Vegetable Image Classification

Name: Tan Wen Tao Bryan
Admin No: P2214449
Class: DAAA/FT/2B/01

---

Project Objective

• To create a CNN for classifying grayscaled images of vegetables and evaluate the performance of the network

Index

1. Exploratory Data Analysis
   
2. Feature Engineering
   
3. Model Selection/Model Evaluation
   
4. Model Improvement
   
5. Summary

References

1. Das, A., 2020, Convolution Neural Network for Image Processing — Using Keras. [online]. Available from: https://towardsdatascience.com/convolution-neural-network-for-image-processing-using-keras-dc3429056306 [Accessed at 5 Nov 2023]
2. M. Israk Ahmed, 2021, DCNN-Based Vegetable Image Classification Using Transfer Learning: A Comparative Study. [online]. Available from: https://www.researchgate.net/publication/352846889_DCNN-Based_Vegetable_Image_Classification_Using_Transfer_Learning_A_Comparative_Study [Accessed at 5 Nov 2023]
3. Bala-Priya, C., 2020, Build Better Deep Learning Models with Batch and Layer Normalization. [online]. Available from: https://www.pinecone.io/learn/batch-layer-normalization/ [Accessed at 17 Nov 2023]
4. Anuja, 2017, L1 and L2 Regularization Methods. [online]. Medium. Available from: https://towardsdatascience.com/l1-and-l2-regularization-methods-ce25e7fc831c [Accessed at 17 Nov 2023]
5. Chi-Feng, W., 2019, The Vanishing Gradient Problem. [online]. Medium. Available from: https://towardsdatascience.com/the-vanishing-gradient-problem-69bf08b15484 [Accessed at 17 Nov 2023]
6. Sanghvirajit, 2021, A Complete Guide to Adam and RMSprop Optimizer. [online]. Medium. Available from: https://medium.com/analytics-vidhya/a-complete-guide-to-adam-and-rmsprop-optimizer-75f4502d83be [Accessed at 17 Nov 2023]
7. Gaudenz, B., 2023, Deep Residual Networks (ResNet, ResNet50) – 2024 Guide [online]. https://viso.ai/deep-learning/resnet-residual-neural-network/ [Accessed at 17 Nov 2023]
8. Salaskar, A., 2021, Quick Start with Tensorflow Callbacks. [online] Available from: https://www.analyticsvidhya.com/blog/2021/08/quick-start-with-tensorflow-callbacks/?utm_source=reading_list&utm_medium=https://www.analyticsvidhya.com/blog/2018/04/fundamentals-deep-learning-regularization-techniques/ [Accessed at 17 Nov 2023]
9. Srivignesh, R., 2021, Tuning Hyperparameters of An Artificial Neural Network Leveraging Keras Tuner [online]. Available from: https://www.analyticsvidhya.com/blog/2021/06/tuning-hyperparameters-of-an-artificial-neural-network-leveraging-keras-tuner/ [Accessed at 17 Nov 2023]
10. datagen.tech, 2015, ResNet-50: The Basics and a Quick Tutorial [online]. Available from: https://datagen.tech/guides/computer-vision/resnet-50/ [Accessed at 17 Nov 2023]
11. Vajira, T., Steven A. Hicks, 2021, Impact of Image Resolution on Deep Learning Performance in Endoscopy Image Classification: An Experimental Study Using a Large Dataset of Endoscopic Images [online]. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8700246/ [Accessed at 17 Nov 2023]

Background Info

Vegetables Image Classification

• DCNN-Based Vegetable Image Classification is used to conduct an experiment on the performance of CNN. This work proposes the study between such typical CNN and its architectures (VGG16, MobileNet, InceptionV3, ResNet etc.) to build up which technique would work best regarding accuracy and effectiveness with new image datasets.
• Additionally, several pre-trained CNN architectures using transfer learning are employed to compare the accuracy with the typical CNN. Study shows that by utilizing previous information gained from related large-scale work, the transfer learning technique can achieve better classification results over traditional CNN with a small dataset.

Convolutional Neural Network

• Take in input image, assign importance (learnable weights and biases) to various aspects in the image, and be able to differentiate one from the other
• Extracts features from the image
• CNN contains the following layers:
  • Input layer (ex. grayscale image)
  • Output layer which is a binary or multi-class labels
  • Hidden layers consisting of convolution layers, ReLU (rectified linear unit) layers, the pooling layers and a fully connected network Image Source: Standford University, 2018

Importing Libraries

# Import libraries
import numpy as np
import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt
import seaborn as sns
import os, math
from sklearn.metrics import classification_report, confusion_matrix, accuracy_score

# Import tensorflow libraries
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Conv2D, MaxPooling2D, GlobalAveragePooling2D ,Flatten, Dropout, Rescaling, BatchNormalization
from tensorflow.keras.applications import ResNet50
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.optimizers import Adam, SGD, RMSprop
from tensorflow.keras.regularizers import l1, l2
from keras_tuner import GridSearch
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.callbacks import LearningRateScheduler, EarlyStopping

# Import Keras libraries
from PIL import Image

Checking GPU

# Check if GPU is available
tf.config.list_physical_devices('GPU')

[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Importing Datasets

# Declare root directory
root_dir = "./Dataset/partA"
train_path = f"{root_dir}/train"
val_path = f"{root_dir}/validation"
test_path = f"{root_dir}/test"

1) Exploratory Data Analysis

Image Visualisation

• Let's see what we are working with

# Retrieve the list of classes
image_categories = os.listdir(train_path)

def plot_images(image_categories):
    plt.figure(figsize=(10, 6))

    for i, cat in enumerate(image_categories):
        image_path = f"{train_path}/{cat}"
        images_inFolder = os.listdir(image_path)
        firstImage_inFolder = images_inFolder[0]
        firstImage_path = f"{image_path}/{firstImage_inFolder}"

        # Load images
        img = tf.keras.utils.load_img(firstImage_path)
        # Normalise the pixel values to the range 0 to 1
        img_arr = tf.keras.utils.img_to_array(img)/255.0

        # Create subplots and plot images
        plt.subplot(3, 5, i+1)
        plt.imshow(img_arr)
        plt.title(cat)
        plt.axis('off')
    plt.tight_layout()
    plt.show()

plot_images(image_categories)

Observations

• There are 15 types of vegetables as labels.
• Data augmentation like image rotation can help for a better prediction.
• Some images like bean and tomato above are quite zoomed out, so zooming in can help with a better prediction.
• Images do not seem to be mislabelled according to the sample of images.

Class Distribution

• Look at the distribution of images for each class in each dataset

def get_label_counts(image_categories, path):
    # Returns classes and their counts
    label_counts={}

    for class_name in image_categories:
        image_path = f"{path}/{class_name}"
        images_inFolder = os.listdir(image_path)

        label_counts[class_name] = len(images_inFolder)
    return label_counts

# Train set
train_label_counts = get_label_counts(image_categories, train_path)

print("No of Labels in Train Set:")
for class_name, count in train_label_counts.items():
    print(f"{class_name}: {count}")

No of Labels in Train Set:
Bean: 780
Bitter_Gourd: 720
Bottle_Gourd: 441
Brinjal: 868
Broccoli: 750
Cabbage: 503
Capsicum: 351
Carrot: 256
Cauliflower: 587
Cucumber: 812
Papaya: 566
Potato: 377
Pumpkin: 814
Radish: 248
Tomato: 955

# Display count in a barplot
class_name = list(train_label_counts.keys())
counts = list(train_label_counts.values())

plt.figure(figsize=(12, 6))
sns.barplot(hue=class_name, y=counts, x=class_name, legend=False, palette="Blues_d")
plt.xlabel("Class Label")
plt.ylabel("Count")
plt.xticks(rotation=45)

plt.show()

# Validation set
validation_label_counts = get_label_counts(image_categories, val_path)

print("No of Labels in Validation Set:")
for class_name, count in validation_label_counts.items():
    print(f"{class_name}: {count}")

No of Labels in Validation Set:
Bean: 200
Bitter_Gourd: 200
Bottle_Gourd: 200
Brinjal: 200
Broccoli: 200
Cabbage: 200
Capsicum: 200
Carrot: 200
Cauliflower: 200
Cucumber: 200
Papaya: 200
Potato: 200
Pumpkin: 200
Radish: 200
Tomato: 200

# Test set
test_label_counts = get_label_counts(image_categories, test_path)

print("No of Labels in Test Set:")
for class_name, count in test_label_counts.items():
    print(f"{class_name}: {count}")

No of Labels in Test Set:
Bean: 200
Bitter_Gourd: 200
Bottle_Gourd: 200
Brinjal: 200
Broccoli: 200
Cabbage: 200
Capsicum: 200
Carrot: 200
Cauliflower: 200
Cucumber: 200
Papaya: 200
Potato: 200
Pumpkin: 200
Radish: 200
Tomato: 200

Observations

• There are unbalanced images in training set
• Equal number of images in validation and testing set

Image Pixel Distribution

• Helps understand how pixel values are distributed across the images

# Function to retrieve the dimensions of the images
def get_pixels(file):
    im = Image.open(file)
    arr = np.array(im)
    h,w,d = arr.shape
    return h, w, d

fig, axes = plt.subplots(5, 3, figsize=(10, 14))
axes = axes.ravel()

for i, class_name in enumerate(image_categories):
    image_path = f"{train_path}/{class_name}"
    fileList = [f"{image_path}/{file}" for file in os.listdir(image_path)]

    # Retrieve the dimensions of the images
    dims = [get_pixels(file) for file in fileList]
    dim_df = pd.DataFrame(dims, columns=["height", "width", "depth"])
    
    # Get the unique dimension values for the current class
    unique_dims = dim_df[["height","width"]].drop_duplicates()
    unique_dims_str = [f"{height}x{width}" for height, width in unique_dims.values]
    print(f"Unique Dimensions for {class_name}:")
    string = "\n".join(unique_dims_str)
    print(string)
    print()

    # Plot the scatterplot to demonstrate the image size
    ax = axes[i]
    sizes = dim_df.groupby(["height","width"]).size().reset_index().rename(columns={0:"count"})
    sizes.plot.scatter(x="height", y="width", ax=ax)
    ax.set_title(f"Image Sizes (pixels) | {class_name}")
    plt.tight_layout()
plt.show()

Unique Dimensions for Bean:
224x224

Unique Dimensions for Bitter_Gourd:
224x224
205x224
200x224

Unique Dimensions for Bottle_Gourd:
224x224

Unique Dimensions for Brinjal:
224x224

Unique Dimensions for Broccoli:
224x224

Unique Dimensions for Cabbage:
224x224

Unique Dimensions for Capsicum:
224x224

Unique Dimensions for Carrot:
224x224

Unique Dimensions for Cauliflower:
224x224

Unique Dimensions for Cucumber:
224x224

Unique Dimensions for Papaya:
224x224
198x224
210x224

Unique Dimensions for Potato:
224x224

Unique Dimensions for Pumpkin:
224x224

Unique Dimensions for Radish:
224x224

Unique Dimensions for Tomato:
224x224

Observation

• Only some images from bitter gourd and papaya has other pixel dimensions.
• The other classes only have images that are 224x224.

Image Averaging

• Calculates the average image for each class within a set of image categories
• Helps in understand the typical appearance of images in each class

# Making (n x m) images
def img2np(path, list_of_filename, size=(64, 64)):
    # Store image arrays
    img_list = []

    for fn in list_of_filename:
        filepath = f"{path}/{fn}"
        # Load images
        current_img = tf.keras.utils.load_img(filepath, target_size=size, color_mode="grayscale")
        img_arr = tf.keras.utils.img_to_array(current_img)
        img_list.append(img_arr)
    
    # Convert to numPy array
    full_mat = np.array(img_list)
    return full_mat

def calculate_avg_img(image_categories, train_dir):
    average_images={}
    for cat in image_categories:
        image_path = f"{train_dir}/{cat}"
        # Retrieve list of images in each class folder
        images_inFolder = os.listdir(image_path)

        # Get the list of images for the current class
        class_images = img2np(image_path, images_inFolder)

        # Get the average image by taking the mean along axis 0
        average_image = np.mean(class_images, axis=0)
        average_images[cat] = average_image

    return average_images

# Calculate the average image for each class
average_images = calculate_avg_img(image_categories, train_path)

# Plot the average images
plt.figure(figsize=(10, 6))
for i, (cat, avg_img) in enumerate(average_images.items()):
    plt.subplot(3, 5, i+1)
    plt.imshow(avg_img.squeeze(), cmap="gray", vmin=0, vmax=255)
    plt.title(f"Average {cat}")
    plt.axis('off')

plt.tight_layout()
plt.show()

Observation

• The average image of each class label is not very clear to view and predict what each image portrays
• Caused by the different orientation of each image, as well as the background of the main object, hence model might have a hard time trying to find features to map to the different images.

Extract Image (Grayscale: 31x31)

• Retrieval of training images of certain dimensions and channel

# Loading data
def load_data(directory, img_height, img_width, colormode, batch_size, seed):
    data = tf.keras.preprocessing.image_dataset_from_directory(
        directory=directory,
        image_size=(img_height, img_width),
        color_mode=colormode,
        batch_size=batch_size, 
        seed = seed
    )
    return data

# Load train data
train_data_31 = load_data(train_path, 31, 31, "grayscale", 10, 42)

Found 9028 files belonging to 15 classes.

# Load validation data
val_data_31 = load_data(val_path, 31, 31, "grayscale", 10, 42)

Found 3000 files belonging to 15 classes.

# Load test data
test_data_31 = load_data(test_path, 31, 31, "grayscale", 10, 42)

Found 3000 files belonging to 15 classes.

Visualising the Dataset

• Randomly showcase a 31x31 grayscale image from each label

def visualise_images(dataset, num_classes, classes, num_images_per_row = 5):
    plt.figure(figsize=(12, 6))
    unique_labels = []
    for image, label in dataset:
        if label[0].numpy() not in unique_labels:
            unique_labels.append(label[0].numpy())
            plt.subplot(num_classes//num_images_per_row, num_images_per_row, len(unique_labels))
            plt.imshow(image[0].numpy().squeeze(), cmap="gray")
            plt.title(f"{classes[label[0].numpy()]}")
            plt.axis('off')

            if len(unique_labels) == num_classes:
                break
    plt.show()

# Determine the number of classes
classes = train_data_31.class_names
num_classes = len(classes)

# Visualise the images
visualise_images(train_data_31, num_classes, classes)

Extract Image (Grayscale: 128x128)

• Retrieval of training images of certain dimensions and channel

# Load train data
train_data_128 = load_data(train_path, 128, 128, "grayscale", 10, 42)

Found 9028 files belonging to 15 classes.

# Load validation data
val_data_128 = load_data(val_path, 128, 128, "grayscale", 10, 42)

Found 3000 files belonging to 15 classes.

# Load test data
test_data_128 = load_data(test_path, 128, 128, "grayscale", 10, 42)

Found 3000 files belonging to 15 classes.

Visualising the Dataset

• Randomly showcase a 128x128 grayscale image from each label

# Determine the number of classes
classes = train_data_128.class_names
num_classes = len(classes)

# Visualise the images
visualise_images(train_data_128, num_classes, classes)

2) Feature Engineering

Normalisation

• Normalise inputs from 0 to 1, leading to faster convergence using the Rescaling layer
• Helps optimize the algorithm to better converge during gradient descent

# Normalise the data within the range of 0 and 1
normalised_data = Sequential(
    name = "normalised_data",
    layers = [
        Rescaling(1./255),  # Normalised values within the range of 0 and 1
    ]
)

This will be added to the first layer of every model.

Loss Function - Sparse Categorical Cross Entropy (One Hot Encoding)

• For multi-class classification, categorical cross entropy is used.
• Instead of one hot encoding, sparse categorical crossentropy is used as the labels are provided as integers:
  • For one hot encoding, each class label is represented as a binary vector with a 1 at the index corresponding to the class and 0s elsewhere. Many classes lead to large, sparse vectors. Example, 100 classes mean 99 zeros and only one 1.
  • For sparse categorical cross entropy, they can handle classification tasks with integer labels directly. Takes the model's predicted probability distribution over classes.

Data Augmentation

• To address the problem of class imbalance, I will augmented the current existing data to generate more training data such that every class will contain the same number of labels.
• Need to reload the train and test data again in batch size of None so all images will get accounted for.

# Reload train data
train_data_31V2 = load_data(train_path, 31, 31, "grayscale", None, 42)

train_data_128V2 = load_data(train_path, 128, 128, "grayscale", None, 42)

Found 9028 files belonging to 15 classes.
Found 9028 files belonging to 15 classes.

# Create a list of images and labels
data_split = [(img.numpy(), classes[labels]) for img, labels in train_data_31V2]

# Convert the list to numpy arrays
X_train_31 = np.array([img for img, label in data_split])
y_train_31 = np.array([label for img, label in data_split])
print(X_train_31.shape)
print(y_train_31.shape)

(9028, 31, 31, 1)
(9028,)

# Create a list of images and labels
data_split = [(img.numpy(), classes[labels]) for img, labels in train_data_128V2]

# Convert the list to numpy arrays
X_train_128 = np.array([img for img, label in data_split])
y_train_128 = np.array([label for img, label in data_split])
print(X_train_128.shape)
print(y_train_128.shape)

(9028, 128, 128, 1)
(9028,)

# Split the data by classes
def split_data_by_classes(X_train, y_train):
    class_data = {}
    for class_label in classes:
        class_indices = np.where(y_train == class_label)[0]
        class_data[class_label] = X_train[class_indices]
    return class_data

# Determine the class max size
def get_max_class_size(class_data):
    max_class_size = max(len(class_images) for class_images in class_data.values())
    print(f"Max class size: {max_class_size}")
    return max_class_size

# Upsample the data to resolve the issue of class imbalance
def upsample_class_with_augmentation(class_data, datagen):
    augmented_images = []
    augmented_labels = []
    max_class_size = get_max_class_size(class_data)
    for class_label, class_images in class_data.items():

        # Set seed for reproducibility to fit the data generator to the images
        datagen.fit(class_images,augment=True, seed=42)

        # Generate augmented images and labels until reaching maximum class size
        generator =datagen.flow(
            class_images, 
            y=np.full((len(class_images),),class_label),
            seed = 42,
            batch_size=len(class_images))
        batch_images, batch_labels = generator.next()
        
        # Repeat data until reaching maximum class size
        while len(batch_images) < max_class_size:
            additional_images, additional_labels = generator.next()
            batch_images = np.concatenate([batch_images, additional_images])
            batch_labels = np.concatenate([batch_labels, additional_labels])

        # Append to lists until max class size is reached
        augmented_images.append(batch_images[:max_class_size])
        augmented_labels.append(batch_labels[:max_class_size])

    # Combine the augmented data for all classes
    augmented_images = np.concatenate(augmented_images)
    augmented_labels = np.concatenate(augmented_labels)
    
    # Shuffle the data
    shuffle_indices = np.arange(len(augmented_labels))
    np.random.seed(42)
    np.random.shuffle(shuffle_indices)

    augmented_images = augmented_images[shuffle_indices]
    augmented_labels = augmented_labels[shuffle_indices]

    return augmented_images, augmented_labels

Solve Class Imbalance - 128 x 128 images

• Types of Augmentation done for 128 x 128 images:
  • Rotation: Allow image to be rotated in the range of -20 to +20 degrees to help the model become invariant to the orientation of objects in the images. Avoid excessive distortion while model learn from images
  • Zoom: Allow image to be zoomed in or out by up to 20% to allow for moderate zooming without distorting the images too much
  • Shear: Shifting one part of the image of a maximum magnitude of 20%, keeping the rest stationary as it introduces deformations to the images, helping the model become more robust to different shapes
  • Horizontal Flip: Images may be horizontally flipped with a 50% probability to increase the variability in the dataset.
  • Fill Mode - Reflect: Smoothly fill in new pixels created during rotations or other transformations, prevent the introduction of sharp edges or artificial patterns

# Augment the images (128 x 128)
datagen = ImageDataGenerator(
    rotation_range=20, # rotation
    zoom_range=0.2, # zoom
    shear_range=0.2, # shear
    horizontal_flip=True, # horizontal flip
    fill_mode = "reflect" # fill mode
)

class_data_128 = split_data_by_classes(X_train_128, y_train_128)
augmented_images_128, augmented_labels_128 = upsample_class_with_augmentation(class_data_128, datagen)

Max class size: 955

# Check number of samples in each class
for class_label, class_images in class_data_128.items():
    num_samples_before_augmentation = len(class_images)
    num_samples_after_augmentation = len(augmented_labels_128[augmented_labels_128 == class_label])
    print(f"Class {class_label}: {num_samples_after_augmentation} samples after augmentation (originally {num_samples_before_augmentation} samples)")

Class Bean: 955 samples after augmentation (originally 780 samples)
Class Bitter_Gourd: 955 samples after augmentation (originally 720 samples)
Class Bottle_Gourd: 955 samples after augmentation (originally 441 samples)
Class Brinjal: 955 samples after augmentation (originally 868 samples)
Class Broccoli: 955 samples after augmentation (originally 750 samples)
Class Cabbage: 955 samples after augmentation (originally 503 samples)
Class Capsicum: 955 samples after augmentation (originally 351 samples)
Class Carrot: 955 samples after augmentation (originally 256 samples)
Class Cauliflower: 955 samples after augmentation (originally 587 samples)
Class Cucumber: 955 samples after augmentation (originally 812 samples)
Class Papaya: 955 samples after augmentation (originally 566 samples)
Class Potato: 955 samples after augmentation (originally 377 samples)
Class Pumpkin: 955 samples after augmentation (originally 814 samples)
Class Radish: 955 samples after augmentation (originally 248 samples)
Class Tomato: 955 samples after augmentation (originally 955 samples)

#Conversion to tensor
class_to_index = {class_name: index for index, class_name in enumerate(classes)}
class_label_int = np.array([class_to_index[label] for label in augmented_labels_128])
train_128V2 = tf.data.Dataset.from_tensor_slices((augmented_images_128, class_label_int))
print(class_label_int.shape)
print(augmented_images_128.shape)

(14325,)
(14325, 128, 128, 1)

# Visualise the first few 20 images
plt.figure(figsize=(15, 10))

for i in range(20):
    plt.subplot(4, 5, i+1)
    plt.imshow(augmented_images_128[i].squeeze(), cmap="gray")
    plt.title(augmented_labels_128[i])
    plt.axis('off')
plt.show()

Solve Class Imbalance - 31 x 31 images

• Rotation: Allow image to be rotated in the range of -10 to +10 degrees to help the model become invariant to the orientation of objects in the images. Avoid excessive distortion while model learn from images
• Shear: Shifting one part of the image of a maximum magnitude of 10%, keeping the rest stationary as it introduces deformations to the images, helping the model become more robust to different shapes
• Horizontal Flip: Images may be horizontally flipped with a 50% probability to increase the variability in the dataset.
• Fill Mode - Reflect: Smoothly fill in new pixels created during rotations or other transformations, prevent the introduction of sharp edges or artificial patterns

# Augment the images (31 x 31)
datagen31 = ImageDataGenerator(
    rotation_range=10, # rotation
    shear_range=0.1, # shear
    horizontal_flip=True, # horizontal flip
    fill_mode = "reflect" # fill mode
)

class_data_31 = split_data_by_classes(X_train_31, y_train_31)
augmented_images_31, augmented_labels_31 = upsample_class_with_augmentation(class_data_31, datagen31)

Max class size: 955

# Check number of samples in each class
for class_label, class_images in class_data_31.items():
    num_samples_before_augmentation = len(class_images)
    num_samples_after_augmentation = len(augmented_labels_31[augmented_labels_31 == class_label])
    print(f"Class {class_label}: {num_samples_after_augmentation} samples after augmentation (originally {num_samples_before_augmentation} samples)")

Class Bean: 955 samples after augmentation (originally 780 samples)
Class Bitter_Gourd: 955 samples after augmentation (originally 720 samples)
Class Bottle_Gourd: 955 samples after augmentation (originally 441 samples)
Class Brinjal: 955 samples after augmentation (originally 868 samples)
Class Broccoli: 955 samples after augmentation (originally 750 samples)
Class Cabbage: 955 samples after augmentation (originally 503 samples)
Class Capsicum: 955 samples after augmentation (originally 351 samples)
Class Carrot: 955 samples after augmentation (originally 256 samples)
Class Cauliflower: 955 samples after augmentation (originally 587 samples)
Class Cucumber: 955 samples after augmentation (originally 812 samples)
Class Papaya: 955 samples after augmentation (originally 566 samples)
Class Potato: 955 samples after augmentation (originally 377 samples)
Class Pumpkin: 955 samples after augmentation (originally 814 samples)
Class Radish: 955 samples after augmentation (originally 248 samples)
Class Tomato: 955 samples after augmentation (originally 955 samples)

#Conversion to tensor
class_to_index = {class_name: index for index, class_name in enumerate(classes)}
class_label_int = np.array([class_to_index[label] for label in augmented_labels_31])
train_31V2 = tf.data.Dataset.from_tensor_slices((augmented_images_31, class_label_int))
print(class_label_int.shape)
print(augmented_images_31.shape)

(14325,)
(14325, 31, 31, 1)

# Visualise the first few 20 images
plt.figure(figsize=(15, 10))

for i in range(20):
    plt.subplot(4, 5, i+1)
    plt.imshow(augmented_images_31[i].squeeze(), cmap="gray")
    plt.title(augmented_labels_31[i])
    plt.axis('off')
plt.show()

3) Model Selection & Model Evaluation

Model Building

Model List (128 x 128 images):

1. Custom Conv2D Neural Network Model

Model List (31 x 31 images):

1. Custom Conv2D Neural Network Model

# Load 128 x 128 data as None to consider all images
train_data_128 = load_data(train_path, 128, 128, "grayscale", None, 42)
val_data_128 = load_data(val_path, 128, 128, "grayscale", None, 42)
test_data_128 = load_data(test_path, 128, 128, "grayscale", None, 42)

Found 9028 files belonging to 15 classes.
Found 3000 files belonging to 15 classes.
Found 3000 files belonging to 15 classes.

# Load 31 x 31 images as batch size None to consider all images
train_data_31 = load_data(train_path, 31, 31, "grayscale", None, 42)
val_data_31 = load_data(val_path, 31, 31, "grayscale", None, 42)
test_data_31 = load_data(test_path, 31, 31, "grayscale", None, 42)

Found 9028 files belonging to 15 classes.
Found 3000 files belonging to 15 classes.
Found 3000 files belonging to 15 classes.

# Plot learning_curve
def plot_learning_curve(history):
    history_df = pd.DataFrame(history)
    epochs = list(range(1,len(history_df)+1))

    fig, ax = plt.subplots(1,2, figsize=(16,6))
    
    # Training loss and validation loss
    ax1=ax[0]
    ax1.plot(epochs, history_df["loss"], label="Training Loss")
    ax1.plot(epochs, history_df["val_loss"], label="Validation Loss")
    ax1.legend()
    ax1.set_ylabel("Loss")
    ax1.set_xlabel("Number of Epochs")
    ax1.set_title("Training and Validation Loss")

    # Training accuracy and validation accuracy
    ax2=ax[1]
    ax2.plot(epochs, history_df["accuracy"], label="Training Accuracy")
    ax2.plot(epochs, history_df["val_accuracy"], label="Validation Accuracy")
    ax2.legend()
    ax2.set_ylabel("Accuracy")
    ax2.set_xlabel("Number of Epochs")
    ax2.set_title("Training and Validation Accuracy")
    plt.show()

Early Stopping Callbacks

• Stop training when validation loss is monitor to stop improving to prevent overfitting
• Min delta is the minimum change in the monitored metric to qualify as an improvement, if change is smaller than 1e-2 (0.01), it will not be considered an improvement.
• Patience - number of epochs with no improvement after which training will be stopped.
• If no improvement in validation loss for 10 consecutive epochs, the training will be stopped.

# Initiate callbacks
callbacks = [
    tf.keras.callbacks.EarlyStopping(
        monitor = "val_loss",
        min_delta=1e-2,
        patience=10,
        verbose=1
    )
]

Conv2D Neural Network Model

• A CNN architecture is well built for image processing as they can effectively extract features from images and learn to recognise patterns, making it suitable for tasks such as object detection, image segmentation and classification.

• Key Layers:

  • Convolutional Layers: Involves sliding a small filter (kernel) over the input to detect features. Conv layers capture local patterns and hierarchical features in the data.
  • Pooling Layers: Downsample the feature maps generated by the convolutional layers. Reduces the spatial dimensions of the data, making the model more robust to variations in position and scale. (Examples: Max-pooling and Average-pooling)
  • Fully Connected Layers: One or more fully connected layers are used to perform high-level reasoning and classification. Layers connect every neuron to every neuron in the previous and subsequent layers, allowing the model to learn complex patterns.
  • Activation Function: Introduces non-linearity into the model. Common functions include: ReLU (Rectified Linear Unit), sigmoid and tanh.
  • Dropout: Regularization technique that randomly drops a fraction of neurons during training to prevent overfitting.
  • Flatten Layer: Used to reshape the data into a 1D vector before passing it to the fully connected layers.
  • Output Layer: Has as many neurons as there are classes. Activation function depends on the problem type. Example, Softmax is commonly used for multi-class classification.
  • Loss Function: Choice of loss function depends on the task. For classification, cross-entropy loss is often used. For regression, mean squared error is commonly used.

• Padding - addition of pixels to the edge of the image

  • Pixels on the edge of the input are only ever exposed to the edge of the filter hence padding can be used to resolved this issue.
  • Padding of value 'same' calculates and adds the padding required to the input image to ensure that the output has the same shape as the input.

• Stride - Amount of movement between applications of the filter to the input image

Training without Data Augumentation

• We will start by fitting the unaugumented data (imbalanced class) to the model as training.

128 x 128 images

Baseline Model (128 x 128)

• To start off, this is my baseline model which I will use to compare the loss and accuracy with the other models.

# Try for 128 x 128 images
tf.keras.backend.clear_session()

Conv2D_128_Baseline = Sequential(
    name = "Conv2D_128_Baseline",
    layers = [
        normalised_data,
        Conv2D(32, (3, 3),input_shape=(128,128,1), activation='relu', padding='same', strides = (4,4)),
        MaxPooling2D((2, 2)),
        Conv2D(64, (3, 3), activation='relu', padding='same', strides=(2,2)),
        MaxPooling2D((2, 2)),
        Conv2D(128, (3, 3), activation='relu', padding='same', strides=(1,1)),
        MaxPooling2D((2, 2)),
        Conv2D(128, (3, 3), activation='relu', padding='same', strides=(1,1)),
        MaxPooling2D((2, 2)),

        Flatten(),

        Dense(128, activation='relu'),
        Dropout(0.6),

        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.0009)
Conv2D_128_Baseline.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_128_Baseline.build(input_shape=(None, 128, 128, 1))

Conv2D_128_Baseline_history = Conv2D_128_Baseline.fit(
    train_data_128.batch(10), 
    epochs=100,
    validation_data=val_data_128.batch(10)
    )

Epoch 1/100
903/903 [==============================] - 6s 6ms/step - loss: 2.4034 - accuracy: 0.1880 - val_loss: 2.0984 - val_accuracy: 0.2967
Epoch 2/100
903/903 [==============================] - 5s 6ms/step - loss: 1.8910 - accuracy: 0.3937 - val_loss: 1.6664 - val_accuracy: 0.4640
Epoch 3/100
903/903 [==============================] - 5s 6ms/step - loss: 1.4737 - accuracy: 0.5332 - val_loss: 1.3177 - val_accuracy: 0.5640
Epoch 4/100
903/903 [==============================] - 5s 6ms/step - loss: 1.0816 - accuracy: 0.6552 - val_loss: 0.9549 - val_accuracy: 0.6987
Epoch 5/100
903/903 [==============================] - 5s 6ms/step - loss: 0.7975 - accuracy: 0.7502 - val_loss: 0.7312 - val_accuracy: 0.7703
Epoch 6/100
903/903 [==============================] - 5s 5ms/step - loss: 0.6206 - accuracy: 0.8079 - val_loss: 0.6366 - val_accuracy: 0.8090
Epoch 7/100
903/903 [==============================] - 5s 6ms/step - loss: 0.4746 - accuracy: 0.8499 - val_loss: 0.6253 - val_accuracy: 0.8190
Epoch 8/100
903/903 [==============================] - 5s 6ms/step - loss: 0.3767 - accuracy: 0.8819 - val_loss: 0.4898 - val_accuracy: 0.8573
Epoch 9/100
903/903 [==============================] - 5s 6ms/step - loss: 0.3425 - accuracy: 0.8954 - val_loss: 0.4888 - val_accuracy: 0.8593
Epoch 10/100
903/903 [==============================] - 5s 6ms/step - loss: 0.2587 - accuracy: 0.9204 - val_loss: 0.5150 - val_accuracy: 0.8623
Epoch 11/100
903/903 [==============================] - 5s 5ms/step - loss: 0.1989 - accuracy: 0.9364 - val_loss: 0.5609 - val_accuracy: 0.8643
Epoch 12/100
903/903 [==============================] - 5s 5ms/step - loss: 0.1993 - accuracy: 0.9373 - val_loss: 0.5555 - val_accuracy: 0.8540
Epoch 13/100
903/903 [==============================] - 5s 5ms/step - loss: 0.1642 - accuracy: 0.9497 - val_loss: 0.5694 - val_accuracy: 0.8673
Epoch 14/100
903/903 [==============================] - 5s 5ms/step - loss: 0.1468 - accuracy: 0.9539 - val_loss: 0.6151 - val_accuracy: 0.8590
Epoch 15/100
903/903 [==============================] - 5s 6ms/step - loss: 0.1559 - accuracy: 0.9546 - val_loss: 0.6875 - val_accuracy: 0.8287
Epoch 16/100
903/903 [==============================] - 5s 6ms/step - loss: 0.1099 - accuracy: 0.9664 - val_loss: 0.5605 - val_accuracy: 0.8747
Epoch 17/100
903/903 [==============================] - 6s 6ms/step - loss: 0.1060 - accuracy: 0.9669 - val_loss: 0.7798 - val_accuracy: 0.8423
Epoch 18/100
903/903 [==============================] - 5s 6ms/step - loss: 0.1008 - accuracy: 0.9700 - val_loss: 0.6616 - val_accuracy: 0.8703
Epoch 19/100
903/903 [==============================] - 5s 6ms/step - loss: 0.1194 - accuracy: 0.9648 - val_loss: 0.7163 - val_accuracy: 0.8410
Epoch 20/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0810 - accuracy: 0.9767 - val_loss: 0.4873 - val_accuracy: 0.8950
Epoch 21/100
903/903 [==============================] - 5s 6ms/step - loss: 0.1003 - accuracy: 0.9703 - val_loss: 0.6490 - val_accuracy: 0.8723
Epoch 22/100
903/903 [==============================] - 5s 6ms/step - loss: 0.1176 - accuracy: 0.9673 - val_loss: 0.5111 - val_accuracy: 0.8900
Epoch 23/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0624 - accuracy: 0.9839 - val_loss: 0.5691 - val_accuracy: 0.8937
Epoch 24/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0815 - accuracy: 0.9762 - val_loss: 0.6889 - val_accuracy: 0.8823
Epoch 25/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0861 - accuracy: 0.9746 - val_loss: 0.8667 - val_accuracy: 0.8437
Epoch 26/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0768 - accuracy: 0.9777 - val_loss: 0.5563 - val_accuracy: 0.9027
Epoch 27/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0620 - accuracy: 0.9834 - val_loss: 0.5861 - val_accuracy: 0.8950
Epoch 28/100
903/903 [==============================] - 5s 6ms/step - loss: 0.1043 - accuracy: 0.9703 - val_loss: 0.5880 - val_accuracy: 0.8853
Epoch 29/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0543 - accuracy: 0.9846 - val_loss: 0.5721 - val_accuracy: 0.9007
Epoch 30/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0728 - accuracy: 0.9801 - val_loss: 0.5965 - val_accuracy: 0.8980
Epoch 31/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0850 - accuracy: 0.9756 - val_loss: 0.7263 - val_accuracy: 0.8813
Epoch 32/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0667 - accuracy: 0.9812 - val_loss: 0.6474 - val_accuracy: 0.8807
Epoch 33/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0570 - accuracy: 0.9846 - val_loss: 0.9009 - val_accuracy: 0.8620
Epoch 34/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0673 - accuracy: 0.9813 - val_loss: 0.6881 - val_accuracy: 0.8927
Epoch 35/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0294 - accuracy: 0.9912 - val_loss: 0.9604 - val_accuracy: 0.8643
Epoch 36/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0985 - accuracy: 0.9746 - val_loss: 0.7437 - val_accuracy: 0.8757
Epoch 37/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0473 - accuracy: 0.9860 - val_loss: 0.8481 - val_accuracy: 0.8660
Epoch 38/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0490 - accuracy: 0.9850 - val_loss: 0.8325 - val_accuracy: 0.8647
Epoch 39/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0513 - accuracy: 0.9870 - val_loss: 1.1974 - val_accuracy: 0.8403
Epoch 40/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0686 - accuracy: 0.9816 - val_loss: 0.7080 - val_accuracy: 0.8840
Epoch 41/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0505 - accuracy: 0.9867 - val_loss: 0.6954 - val_accuracy: 0.8950
Epoch 42/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0441 - accuracy: 0.9872 - val_loss: 0.6078 - val_accuracy: 0.9050
Epoch 43/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0610 - accuracy: 0.9845 - val_loss: 0.7151 - val_accuracy: 0.8930
Epoch 44/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0476 - accuracy: 0.9858 - val_loss: 0.8962 - val_accuracy: 0.8807
Epoch 45/100
903/903 [==============================] - 6s 6ms/step - loss: 0.0480 - accuracy: 0.9874 - val_loss: 0.9012 - val_accuracy: 0.8790
Epoch 46/100
903/903 [==============================] - 6s 6ms/step - loss: 0.0555 - accuracy: 0.9854 - val_loss: 0.6981 - val_accuracy: 0.8927
Epoch 47/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0607 - accuracy: 0.9854 - val_loss: 0.8116 - val_accuracy: 0.8797
Epoch 48/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0606 - accuracy: 0.9858 - val_loss: 0.7625 - val_accuracy: 0.8810
Epoch 49/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0417 - accuracy: 0.9879 - val_loss: 0.8140 - val_accuracy: 0.8823
Epoch 50/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0609 - accuracy: 0.9828 - val_loss: 0.8827 - val_accuracy: 0.8873
Epoch 51/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0465 - accuracy: 0.9874 - val_loss: 0.8965 - val_accuracy: 0.8700
Epoch 52/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0498 - accuracy: 0.9874 - val_loss: 0.8028 - val_accuracy: 0.8993
Epoch 53/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0514 - accuracy: 0.9877 - val_loss: 1.0280 - val_accuracy: 0.8687
Epoch 54/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0365 - accuracy: 0.9900 - val_loss: 1.9454 - val_accuracy: 0.7830
Epoch 55/100
903/903 [==============================] - 6s 6ms/step - loss: 0.0411 - accuracy: 0.9897 - val_loss: 0.9099 - val_accuracy: 0.8850
Epoch 56/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0354 - accuracy: 0.9912 - val_loss: 0.7715 - val_accuracy: 0.8973
Epoch 57/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0512 - accuracy: 0.9887 - val_loss: 1.3893 - val_accuracy: 0.8500
Epoch 58/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0656 - accuracy: 0.9831 - val_loss: 0.8867 - val_accuracy: 0.8817
Epoch 59/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0392 - accuracy: 0.9893 - val_loss: 1.1628 - val_accuracy: 0.8620
Epoch 60/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0724 - accuracy: 0.9809 - val_loss: 1.2760 - val_accuracy: 0.8453
Epoch 61/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0542 - accuracy: 0.9860 - val_loss: 0.8908 - val_accuracy: 0.8733
Epoch 62/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0503 - accuracy: 0.9887 - val_loss: 0.9616 - val_accuracy: 0.8917
Epoch 63/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0392 - accuracy: 0.9905 - val_loss: 0.8718 - val_accuracy: 0.8900
Epoch 64/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0470 - accuracy: 0.9883 - val_loss: 1.1139 - val_accuracy: 0.8490
Epoch 65/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0627 - accuracy: 0.9836 - val_loss: 0.8728 - val_accuracy: 0.8893
Epoch 66/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0315 - accuracy: 0.9922 - val_loss: 0.8944 - val_accuracy: 0.8963
Epoch 67/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0308 - accuracy: 0.9922 - val_loss: 1.4803 - val_accuracy: 0.8567
Epoch 68/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0451 - accuracy: 0.9889 - val_loss: 0.9810 - val_accuracy: 0.8897
Epoch 69/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0398 - accuracy: 0.9906 - val_loss: 0.9094 - val_accuracy: 0.8900
Epoch 70/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0506 - accuracy: 0.9874 - val_loss: 0.9607 - val_accuracy: 0.8987
Epoch 71/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0439 - accuracy: 0.9891 - val_loss: 1.0190 - val_accuracy: 0.8737
Epoch 72/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0437 - accuracy: 0.9896 - val_loss: 1.0406 - val_accuracy: 0.8867
Epoch 73/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0705 - accuracy: 0.9854 - val_loss: 0.8938 - val_accuracy: 0.8947
Epoch 74/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0542 - accuracy: 0.9866 - val_loss: 1.0079 - val_accuracy: 0.8770
Epoch 75/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0134 - accuracy: 0.9961 - val_loss: 0.8737 - val_accuracy: 0.9043
Epoch 76/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0138 - accuracy: 0.9963 - val_loss: 1.3317 - val_accuracy: 0.8703
Epoch 77/100
903/903 [==============================] - 6s 6ms/step - loss: 0.0785 - accuracy: 0.9833 - val_loss: 0.9487 - val_accuracy: 0.8823
Epoch 78/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0497 - accuracy: 0.9886 - val_loss: 1.0737 - val_accuracy: 0.8890
Epoch 79/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0414 - accuracy: 0.9912 - val_loss: 1.2216 - val_accuracy: 0.8840
Epoch 80/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0434 - accuracy: 0.9890 - val_loss: 1.1579 - val_accuracy: 0.8750
Epoch 81/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0383 - accuracy: 0.9912 - val_loss: 0.9789 - val_accuracy: 0.8947
Epoch 82/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0495 - accuracy: 0.9875 - val_loss: 1.0354 - val_accuracy: 0.8913
Epoch 83/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0285 - accuracy: 0.9930 - val_loss: 1.1209 - val_accuracy: 0.8917
Epoch 84/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0505 - accuracy: 0.9893 - val_loss: 1.2365 - val_accuracy: 0.8750
Epoch 85/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0510 - accuracy: 0.9893 - val_loss: 1.0503 - val_accuracy: 0.8760
Epoch 86/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0439 - accuracy: 0.9909 - val_loss: 1.0251 - val_accuracy: 0.8897
Epoch 87/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0278 - accuracy: 0.9930 - val_loss: 0.9666 - val_accuracy: 0.8990
Epoch 88/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0510 - accuracy: 0.9901 - val_loss: 1.0432 - val_accuracy: 0.8883
Epoch 89/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0469 - accuracy: 0.9906 - val_loss: 1.0915 - val_accuracy: 0.8843
Epoch 90/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0416 - accuracy: 0.9896 - val_loss: 1.1019 - val_accuracy: 0.8820
Epoch 91/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0412 - accuracy: 0.9904 - val_loss: 1.7373 - val_accuracy: 0.8410
Epoch 92/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0453 - accuracy: 0.9907 - val_loss: 1.2863 - val_accuracy: 0.8757
Epoch 93/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0364 - accuracy: 0.9918 - val_loss: 1.2700 - val_accuracy: 0.8823
Epoch 94/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0233 - accuracy: 0.9941 - val_loss: 1.2311 - val_accuracy: 0.8920
Epoch 95/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0364 - accuracy: 0.9910 - val_loss: 1.3214 - val_accuracy: 0.8780
Epoch 96/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0399 - accuracy: 0.9917 - val_loss: 1.4522 - val_accuracy: 0.8690
Epoch 97/100
903/903 [==============================] - 5s 6ms/step - loss: 0.0738 - accuracy: 0.9869 - val_loss: 1.1717 - val_accuracy: 0.8893
Epoch 98/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0363 - accuracy: 0.9930 - val_loss: 1.1103 - val_accuracy: 0.8687
Epoch 99/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0427 - accuracy: 0.9906 - val_loss: 1.3007 - val_accuracy: 0.8743
Epoch 100/100
903/903 [==============================] - 5s 5ms/step - loss: 0.0353 - accuracy: 0.9926 - val_loss: 1.3019 - val_accuracy: 0.8837

Conv2D_128_Baseline.summary()

Model: "Conv2D_128_Baseline"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalised_data (Sequential  (None, None, None, 1)    0         
 )                                                               
                                                                 
 conv2d (Conv2D)             (None, 32, 32, 32)        320       
                                                                 
 max_pooling2d (MaxPooling2D  (None, 16, 16, 32)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 8, 8, 64)          18496     
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 4, 4, 64)         0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 4, 4, 128)         73856     
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 2, 2, 128)        0         
 2D)                                                             
                                                                 
 conv2d_3 (Conv2D)           (None, 2, 2, 128)         147584    
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 1, 1, 128)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 128)               0         
                                                                 
 dense (Dense)               (None, 128)               16512     
                                                                 
 dropout (Dropout)           (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 15)                1935      
                                                                 
=================================================================
Total params: 258,703
Trainable params: 258,703
Non-trainable params: 0
_________________________________________________________________

Observations

• Model has 258703 trainable parameters.
• Model has 4 Convolutional Layers, 4 Max Pooling layers, 1 Flatten Layer, 1 Droput Layer, 1 Fully Connected Layer and 1 Output Layer.

plot_learning_curve(Conv2D_128_Baseline_history.history)

Observations

• From the loss curve, as the epochs increased, the loss function generally starts to increase.
• From the accuracy curve, as the epochs increased, the accuracy is fluctuating around 0.80+.
• Both graphs imply that the learning rate might be too high, causing the model to overshoot the minimum of the loss function due to the major fluctuations present in both curves.

Conv2D_128_Baseline.evaluate(test_data_128.batch(10))

300/300 [==============================] - 1s 3ms/step - loss: 1.1186 - accuracy: 0.8940

[1.1186254024505615, 0.8939999938011169]

Observations

• This CNN model generated an average loss of 0.88 and an accuracy of 0.86+ when evaluated on test dataset, it can be improved.

CNN Version 2 (128 x 128)

• Next attempt to build a model with a lower learning rate, add in strides and more regularisation from the use of the DropOut layers

# Try for 128 x 128 images
tf.keras.backend.clear_session()

Conv2D_128_V2 = Sequential(
    name = "Conv2D_128_V2",
    layers = [
        normalised_data,
        Conv2D(64, (5,5),input_shape=(128,128,1), activation = "relu", padding='same', strides = (4,4)),
        MaxPooling2D((2, 2)),
        Conv2D(128, (5, 5), activation = "relu", padding='same', strides = (2, 2)),
        MaxPooling2D((2, 2)),
        Conv2D(256, (3,3), activation = "relu", padding='same', strides = (1, 1)),
        MaxPooling2D((2, 2)),
        Conv2D(512, (3,3), activation = "relu", padding='same', strides = (1, 1)),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.5),

        Dense(512, activation='relu'),
        Dropout(0.5),
        Dense(128, activation='relu'),
        Dropout(0.5),

        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.00001)
Conv2D_128_V2.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_128_V2.build(input_shape=(None, 128, 128, 1))

Conv2D_128_V2_history = Conv2D_128_V2.fit(
    train_data_128.batch(10), 
    epochs=300,
    validation_data=val_data_128.batch(10)
    )

Epoch 1/300
903/903 [==============================] - 6s 7ms/step - loss: 2.6847 - accuracy: 0.0892 - val_loss: 2.7196 - val_accuracy: 0.0667
Epoch 2/300
903/903 [==============================] - 6s 6ms/step - loss: 2.6574 - accuracy: 0.0969 - val_loss: 2.7282 - val_accuracy: 0.0667
Epoch 3/300
903/903 [==============================] - 6s 6ms/step - loss: 2.6561 - accuracy: 0.1005 - val_loss: 2.7235 - val_accuracy: 0.0667
Epoch 4/300
903/903 [==============================] - 6s 6ms/step - loss: 2.6480 - accuracy: 0.1027 - val_loss: 2.7204 - val_accuracy: 0.0667
Epoch 5/300
903/903 [==============================] - 6s 6ms/step - loss: 2.6302 - accuracy: 0.1031 - val_loss: 2.7059 - val_accuracy: 0.0680
Epoch 6/300
903/903 [==============================] - 6s 6ms/step - loss: 2.6000 - accuracy: 0.1133 - val_loss: 2.6450 - val_accuracy: 0.1260
Epoch 7/300
903/903 [==============================] - 6s 6ms/step - loss: 2.5494 - accuracy: 0.1331 - val_loss: 2.5761 - val_accuracy: 0.1270
Epoch 8/300
903/903 [==============================] - 6s 6ms/step - loss: 2.5073 - accuracy: 0.1520 - val_loss: 2.5371 - val_accuracy: 0.1297
Epoch 9/300
903/903 [==============================] - 6s 6ms/step - loss: 2.4705 - accuracy: 0.1618 - val_loss: 2.5062 - val_accuracy: 0.1523
Epoch 10/300
903/903 [==============================] - 6s 6ms/step - loss: 2.4468 - accuracy: 0.1733 - val_loss: 2.4815 - val_accuracy: 0.1540
Epoch 11/300
903/903 [==============================] - 6s 6ms/step - loss: 2.4291 - accuracy: 0.1709 - val_loss: 2.4576 - val_accuracy: 0.1640
Epoch 12/300
903/903 [==============================] - 6s 6ms/step - loss: 2.4069 - accuracy: 0.1851 - val_loss: 2.4304 - val_accuracy: 0.1793
Epoch 13/300
903/903 [==============================] - 6s 6ms/step - loss: 2.3749 - accuracy: 0.1967 - val_loss: 2.3966 - val_accuracy: 0.2080
Epoch 14/300
903/903 [==============================] - 6s 6ms/step - loss: 2.3544 - accuracy: 0.1998 - val_loss: 2.3655 - val_accuracy: 0.2170
Epoch 15/300
903/903 [==============================] - 6s 6ms/step - loss: 2.3302 - accuracy: 0.2107 - val_loss: 2.3536 - val_accuracy: 0.2263
Epoch 16/300
903/903 [==============================] - 6s 6ms/step - loss: 2.2948 - accuracy: 0.2274 - val_loss: 2.2953 - val_accuracy: 0.2620
Epoch 17/300
903/903 [==============================] - 6s 6ms/step - loss: 2.2652 - accuracy: 0.2446 - val_loss: 2.2613 - val_accuracy: 0.2833
Epoch 18/300
903/903 [==============================] - 6s 6ms/step - loss: 2.2398 - accuracy: 0.2480 - val_loss: 2.2301 - val_accuracy: 0.2913
Epoch 19/300
903/903 [==============================] - 6s 6ms/step - loss: 2.1960 - accuracy: 0.2595 - val_loss: 2.1838 - val_accuracy: 0.3020
Epoch 20/300
903/903 [==============================] - 6s 6ms/step - loss: 2.1801 - accuracy: 0.2710 - val_loss: 2.1531 - val_accuracy: 0.3120
Epoch 21/300
903/903 [==============================] - 6s 6ms/step - loss: 2.1484 - accuracy: 0.2842 - val_loss: 2.1415 - val_accuracy: 0.3143
Epoch 22/300
903/903 [==============================] - 6s 6ms/step - loss: 2.1179 - accuracy: 0.2955 - val_loss: 2.1016 - val_accuracy: 0.3343
Epoch 23/300
903/903 [==============================] - 6s 6ms/step - loss: 2.0814 - accuracy: 0.3083 - val_loss: 2.0933 - val_accuracy: 0.3277
Epoch 24/300
903/903 [==============================] - 6s 6ms/step - loss: 2.0539 - accuracy: 0.3187 - val_loss: 2.0394 - val_accuracy: 0.3417
Epoch 25/300
903/903 [==============================] - 6s 6ms/step - loss: 2.0405 - accuracy: 0.3282 - val_loss: 2.0156 - val_accuracy: 0.3483
Epoch 26/300
903/903 [==============================] - 6s 6ms/step - loss: 2.0213 - accuracy: 0.3319 - val_loss: 2.0055 - val_accuracy: 0.3497
Epoch 27/300
903/903 [==============================] - 6s 6ms/step - loss: 1.9971 - accuracy: 0.3402 - val_loss: 1.9602 - val_accuracy: 0.3677
Epoch 28/300
903/903 [==============================] - 6s 6ms/step - loss: 1.9699 - accuracy: 0.3465 - val_loss: 1.9422 - val_accuracy: 0.3730
Epoch 29/300
903/903 [==============================] - 6s 6ms/step - loss: 1.9559 - accuracy: 0.3514 - val_loss: 1.9143 - val_accuracy: 0.3723
Epoch 30/300
903/903 [==============================] - 6s 6ms/step - loss: 1.9240 - accuracy: 0.3590 - val_loss: 1.8836 - val_accuracy: 0.3937
Epoch 31/300
903/903 [==============================] - 6s 6ms/step - loss: 1.9035 - accuracy: 0.3733 - val_loss: 1.8800 - val_accuracy: 0.3850
Epoch 32/300
903/903 [==============================] - 6s 6ms/step - loss: 1.8806 - accuracy: 0.3802 - val_loss: 1.8823 - val_accuracy: 0.3833
Epoch 33/300
903/903 [==============================] - 6s 6ms/step - loss: 1.8628 - accuracy: 0.3849 - val_loss: 1.8259 - val_accuracy: 0.4050
Epoch 34/300
903/903 [==============================] - 6s 6ms/step - loss: 1.8449 - accuracy: 0.3854 - val_loss: 1.8142 - val_accuracy: 0.4067
Epoch 35/300
903/903 [==============================] - 6s 6ms/step - loss: 1.8255 - accuracy: 0.4002 - val_loss: 1.7730 - val_accuracy: 0.4200
Epoch 36/300
903/903 [==============================] - 6s 6ms/step - loss: 1.8086 - accuracy: 0.4043 - val_loss: 1.7609 - val_accuracy: 0.4203
Epoch 37/300
903/903 [==============================] - 6s 6ms/step - loss: 1.7902 - accuracy: 0.4111 - val_loss: 1.7407 - val_accuracy: 0.4320
Epoch 38/300
903/903 [==============================] - 6s 6ms/step - loss: 1.7687 - accuracy: 0.4101 - val_loss: 1.7265 - val_accuracy: 0.4280
Epoch 39/300
903/903 [==============================] - 6s 6ms/step - loss: 1.7486 - accuracy: 0.4235 - val_loss: 1.7133 - val_accuracy: 0.4350
Epoch 40/300
903/903 [==============================] - 6s 6ms/step - loss: 1.7210 - accuracy: 0.4310 - val_loss: 1.7297 - val_accuracy: 0.4417
Epoch 41/300
903/903 [==============================] - 6s 6ms/step - loss: 1.7145 - accuracy: 0.4288 - val_loss: 1.6661 - val_accuracy: 0.4503
Epoch 42/300
903/903 [==============================] - 6s 6ms/step - loss: 1.6814 - accuracy: 0.4449 - val_loss: 1.6385 - val_accuracy: 0.4670
Epoch 43/300
903/903 [==============================] - 6s 6ms/step - loss: 1.6551 - accuracy: 0.4496 - val_loss: 1.6107 - val_accuracy: 0.4753
Epoch 44/300
903/903 [==============================] - 6s 6ms/step - loss: 1.6512 - accuracy: 0.4609 - val_loss: 1.6011 - val_accuracy: 0.4797
Epoch 45/300
903/903 [==============================] - 6s 6ms/step - loss: 1.6265 - accuracy: 0.4587 - val_loss: 1.6477 - val_accuracy: 0.4557
Epoch 46/300
903/903 [==============================] - 6s 6ms/step - loss: 1.6228 - accuracy: 0.4652 - val_loss: 1.5910 - val_accuracy: 0.4903
Epoch 47/300
903/903 [==============================] - 6s 6ms/step - loss: 1.5883 - accuracy: 0.4752 - val_loss: 1.5718 - val_accuracy: 0.4860
Epoch 48/300
903/903 [==============================] - 6s 7ms/step - loss: 1.5718 - accuracy: 0.4821 - val_loss: 1.5175 - val_accuracy: 0.5080
Epoch 49/300
903/903 [==============================] - 6s 7ms/step - loss: 1.5550 - accuracy: 0.4911 - val_loss: 1.5039 - val_accuracy: 0.5117
Epoch 50/300
903/903 [==============================] - 6s 7ms/step - loss: 1.5418 - accuracy: 0.4949 - val_loss: 1.4969 - val_accuracy: 0.5080
Epoch 51/300
903/903 [==============================] - 6s 6ms/step - loss: 1.5175 - accuracy: 0.5056 - val_loss: 1.4927 - val_accuracy: 0.5110
Epoch 52/300
903/903 [==============================] - 6s 6ms/step - loss: 1.4857 - accuracy: 0.5088 - val_loss: 1.4636 - val_accuracy: 0.5160
Epoch 53/300
903/903 [==============================] - 6s 6ms/step - loss: 1.4712 - accuracy: 0.5195 - val_loss: 1.4287 - val_accuracy: 0.5280
Epoch 54/300
903/903 [==============================] - 6s 7ms/step - loss: 1.4550 - accuracy: 0.5226 - val_loss: 1.4212 - val_accuracy: 0.5310
Epoch 55/300
903/903 [==============================] - 6s 7ms/step - loss: 1.4463 - accuracy: 0.5304 - val_loss: 1.3899 - val_accuracy: 0.5407
Epoch 56/300
903/903 [==============================] - 6s 7ms/step - loss: 1.4150 - accuracy: 0.5340 - val_loss: 1.4266 - val_accuracy: 0.5257
Epoch 57/300
903/903 [==============================] - 6s 7ms/step - loss: 1.3986 - accuracy: 0.5460 - val_loss: 1.3576 - val_accuracy: 0.5523
Epoch 58/300
903/903 [==============================] - 6s 6ms/step - loss: 1.3828 - accuracy: 0.5488 - val_loss: 1.3466 - val_accuracy: 0.5607
Epoch 59/300
903/903 [==============================] - 6s 6ms/step - loss: 1.3614 - accuracy: 0.5537 - val_loss: 1.3609 - val_accuracy: 0.5520
Epoch 60/300
903/903 [==============================] - 6s 6ms/step - loss: 1.3288 - accuracy: 0.5677 - val_loss: 1.3020 - val_accuracy: 0.5747
Epoch 61/300
903/903 [==============================] - 6s 6ms/step - loss: 1.3212 - accuracy: 0.5714 - val_loss: 1.3173 - val_accuracy: 0.5593
Epoch 62/300
903/903 [==============================] - 6s 6ms/step - loss: 1.3087 - accuracy: 0.5727 - val_loss: 1.2962 - val_accuracy: 0.5857
Epoch 63/300
903/903 [==============================] - 6s 6ms/step - loss: 1.3011 - accuracy: 0.5747 - val_loss: 1.2520 - val_accuracy: 0.5977
Epoch 64/300
903/903 [==============================] - 6s 6ms/step - loss: 1.2735 - accuracy: 0.5839 - val_loss: 1.2429 - val_accuracy: 0.6013
Epoch 65/300
903/903 [==============================] - 6s 6ms/step - loss: 1.2535 - accuracy: 0.5929 - val_loss: 1.2305 - val_accuracy: 0.6040
Epoch 66/300
903/903 [==============================] - 6s 6ms/step - loss: 1.2382 - accuracy: 0.5939 - val_loss: 1.2067 - val_accuracy: 0.6083
Epoch 67/300
903/903 [==============================] - 6s 6ms/step - loss: 1.2071 - accuracy: 0.6077 - val_loss: 1.2030 - val_accuracy: 0.6080
Epoch 68/300
903/903 [==============================] - 6s 6ms/step - loss: 1.1941 - accuracy: 0.6083 - val_loss: 1.1684 - val_accuracy: 0.6177
Epoch 69/300
903/903 [==============================] - 6s 6ms/step - loss: 1.1838 - accuracy: 0.6153 - val_loss: 1.1894 - val_accuracy: 0.6057
Epoch 70/300
903/903 [==============================] - 6s 6ms/step - loss: 1.1679 - accuracy: 0.6196 - val_loss: 1.1420 - val_accuracy: 0.6343
Epoch 71/300
903/903 [==============================] - 6s 6ms/step - loss: 1.1421 - accuracy: 0.6335 - val_loss: 1.1235 - val_accuracy: 0.6377
Epoch 72/300
903/903 [==============================] - 6s 7ms/step - loss: 1.1377 - accuracy: 0.6273 - val_loss: 1.1160 - val_accuracy: 0.6320
Epoch 73/300
903/903 [==============================] - 6s 6ms/step - loss: 1.1140 - accuracy: 0.6407 - val_loss: 1.0919 - val_accuracy: 0.6457
Epoch 74/300
903/903 [==============================] - 6s 7ms/step - loss: 1.1056 - accuracy: 0.6400 - val_loss: 1.0817 - val_accuracy: 0.6570
Epoch 75/300
903/903 [==============================] - 6s 6ms/step - loss: 1.0804 - accuracy: 0.6520 - val_loss: 1.1100 - val_accuracy: 0.6440
Epoch 76/300
903/903 [==============================] - 6s 6ms/step - loss: 1.0708 - accuracy: 0.6577 - val_loss: 1.1161 - val_accuracy: 0.6327
Epoch 77/300
903/903 [==============================] - 6s 6ms/step - loss: 1.0582 - accuracy: 0.6627 - val_loss: 1.1601 - val_accuracy: 0.6150
Epoch 78/300
903/903 [==============================] - 6s 6ms/step - loss: 1.0483 - accuracy: 0.6673 - val_loss: 1.0401 - val_accuracy: 0.6580
Epoch 79/300
903/903 [==============================] - 6s 7ms/step - loss: 1.0176 - accuracy: 0.6778 - val_loss: 1.0081 - val_accuracy: 0.6773
Epoch 80/300
903/903 [==============================] - 8s 9ms/step - loss: 1.0098 - accuracy: 0.6743 - val_loss: 1.0272 - val_accuracy: 0.6613
Epoch 81/300
903/903 [==============================] - 6s 6ms/step - loss: 0.9988 - accuracy: 0.6810 - val_loss: 1.0115 - val_accuracy: 0.6670
Epoch 82/300
903/903 [==============================] - 6s 7ms/step - loss: 0.9851 - accuracy: 0.6859 - val_loss: 1.0017 - val_accuracy: 0.6587
Epoch 83/300
903/903 [==============================] - 6s 7ms/step - loss: 0.9562 - accuracy: 0.6924 - val_loss: 0.9721 - val_accuracy: 0.6917
Epoch 84/300
903/903 [==============================] - 6s 7ms/step - loss: 0.9557 - accuracy: 0.6977 - val_loss: 0.9841 - val_accuracy: 0.6763
Epoch 85/300
903/903 [==============================] - 6s 7ms/step - loss: 0.9487 - accuracy: 0.6995 - val_loss: 0.9751 - val_accuracy: 0.6903
Epoch 86/300
903/903 [==============================] - 6s 7ms/step - loss: 0.9431 - accuracy: 0.6997 - val_loss: 0.9613 - val_accuracy: 0.7067
Epoch 87/300
903/903 [==============================] - 6s 7ms/step - loss: 0.9222 - accuracy: 0.7064 - val_loss: 0.9400 - val_accuracy: 0.6957
Epoch 88/300
903/903 [==============================] - 6s 7ms/step - loss: 0.9028 - accuracy: 0.7117 - val_loss: 0.9138 - val_accuracy: 0.7110
Epoch 89/300
903/903 [==============================] - 6s 7ms/step - loss: 0.8834 - accuracy: 0.7109 - val_loss: 0.9315 - val_accuracy: 0.7047
Epoch 90/300
903/903 [==============================] - 7s 7ms/step - loss: 0.8873 - accuracy: 0.7223 - val_loss: 0.9043 - val_accuracy: 0.7133
Epoch 91/300
903/903 [==============================] - 7s 7ms/step - loss: 0.8627 - accuracy: 0.7235 - val_loss: 0.8850 - val_accuracy: 0.7257
Epoch 92/300
903/903 [==============================] - 7s 7ms/step - loss: 0.8523 - accuracy: 0.7280 - val_loss: 0.8639 - val_accuracy: 0.7297
Epoch 93/300
903/903 [==============================] - 6s 7ms/step - loss: 0.8558 - accuracy: 0.7315 - val_loss: 0.8629 - val_accuracy: 0.7330
Epoch 94/300
903/903 [==============================] - 6s 7ms/step - loss: 0.8249 - accuracy: 0.7362 - val_loss: 0.8407 - val_accuracy: 0.7447
Epoch 95/300
903/903 [==============================] - 6s 7ms/step - loss: 0.8123 - accuracy: 0.7370 - val_loss: 0.8395 - val_accuracy: 0.7337
Epoch 96/300
903/903 [==============================] - 6s 7ms/step - loss: 0.8121 - accuracy: 0.7396 - val_loss: 0.8456 - val_accuracy: 0.7393
Epoch 97/300
903/903 [==============================] - 6s 7ms/step - loss: 0.7966 - accuracy: 0.7430 - val_loss: 0.8282 - val_accuracy: 0.7447
Epoch 98/300
903/903 [==============================] - 6s 6ms/step - loss: 0.7782 - accuracy: 0.7527 - val_loss: 0.8374 - val_accuracy: 0.7470
Epoch 99/300
903/903 [==============================] - 6s 7ms/step - loss: 0.7745 - accuracy: 0.7544 - val_loss: 0.7908 - val_accuracy: 0.7560
Epoch 100/300
903/903 [==============================] - 6s 6ms/step - loss: 0.7697 - accuracy: 0.7529 - val_loss: 0.8111 - val_accuracy: 0.7530
Epoch 101/300
903/903 [==============================] - 6s 6ms/step - loss: 0.7489 - accuracy: 0.7622 - val_loss: 0.8036 - val_accuracy: 0.7567
Epoch 102/300
903/903 [==============================] - 6s 6ms/step - loss: 0.7380 - accuracy: 0.7653 - val_loss: 0.7760 - val_accuracy: 0.7647
Epoch 103/300
903/903 [==============================] - 6s 6ms/step - loss: 0.7299 - accuracy: 0.7730 - val_loss: 0.8174 - val_accuracy: 0.7533
Epoch 104/300
903/903 [==============================] - 6s 6ms/step - loss: 0.7195 - accuracy: 0.7684 - val_loss: 0.7631 - val_accuracy: 0.7730
Epoch 105/300
903/903 [==============================] - 6s 6ms/step - loss: 0.7099 - accuracy: 0.7723 - val_loss: 0.7572 - val_accuracy: 0.7730
Epoch 106/300
903/903 [==============================] - 6s 6ms/step - loss: 0.7043 - accuracy: 0.7784 - val_loss: 0.7571 - val_accuracy: 0.7703
Epoch 107/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6866 - accuracy: 0.7779 - val_loss: 0.7393 - val_accuracy: 0.7743
Epoch 108/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6762 - accuracy: 0.7836 - val_loss: 0.7670 - val_accuracy: 0.7633
Epoch 109/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6783 - accuracy: 0.7901 - val_loss: 0.7259 - val_accuracy: 0.7837
Epoch 110/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6685 - accuracy: 0.7911 - val_loss: 0.7169 - val_accuracy: 0.7860
Epoch 111/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6439 - accuracy: 0.7976 - val_loss: 0.7184 - val_accuracy: 0.7710
Epoch 112/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6318 - accuracy: 0.8025 - val_loss: 0.7197 - val_accuracy: 0.7833
Epoch 113/300
903/903 [==============================] - 6s 7ms/step - loss: 0.6287 - accuracy: 0.8027 - val_loss: 0.6837 - val_accuracy: 0.7897
Epoch 114/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6210 - accuracy: 0.8057 - val_loss: 0.7258 - val_accuracy: 0.7640
Epoch 115/300
903/903 [==============================] - 6s 7ms/step - loss: 0.6189 - accuracy: 0.8080 - val_loss: 0.6785 - val_accuracy: 0.7903
Epoch 116/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6178 - accuracy: 0.8035 - val_loss: 0.6719 - val_accuracy: 0.7980
Epoch 117/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6061 - accuracy: 0.8089 - val_loss: 0.6653 - val_accuracy: 0.7997
Epoch 118/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5780 - accuracy: 0.8201 - val_loss: 0.6694 - val_accuracy: 0.7943
Epoch 119/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5798 - accuracy: 0.8161 - val_loss: 0.6906 - val_accuracy: 0.7863
Epoch 120/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5665 - accuracy: 0.8251 - val_loss: 0.7159 - val_accuracy: 0.7730
Epoch 121/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5639 - accuracy: 0.8217 - val_loss: 0.6612 - val_accuracy: 0.7927
Epoch 122/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5582 - accuracy: 0.8255 - val_loss: 0.6511 - val_accuracy: 0.8007
Epoch 123/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5432 - accuracy: 0.8284 - val_loss: 0.6528 - val_accuracy: 0.8043
Epoch 124/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5432 - accuracy: 0.8320 - val_loss: 0.6696 - val_accuracy: 0.7947
Epoch 125/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5288 - accuracy: 0.8344 - val_loss: 0.6259 - val_accuracy: 0.8103
Epoch 126/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5245 - accuracy: 0.8377 - val_loss: 0.6477 - val_accuracy: 0.7983
Epoch 127/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5186 - accuracy: 0.8393 - val_loss: 0.6193 - val_accuracy: 0.8097
Epoch 128/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4950 - accuracy: 0.8459 - val_loss: 0.6258 - val_accuracy: 0.8100
Epoch 129/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4887 - accuracy: 0.8463 - val_loss: 0.6274 - val_accuracy: 0.8093
Epoch 130/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4994 - accuracy: 0.8440 - val_loss: 0.5843 - val_accuracy: 0.8213
Epoch 131/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4818 - accuracy: 0.8479 - val_loss: 0.6106 - val_accuracy: 0.8207
Epoch 132/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4727 - accuracy: 0.8507 - val_loss: 0.6360 - val_accuracy: 0.8083
Epoch 133/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4668 - accuracy: 0.8543 - val_loss: 0.5914 - val_accuracy: 0.8230
Epoch 134/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4518 - accuracy: 0.8559 - val_loss: 0.5810 - val_accuracy: 0.8223
Epoch 135/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4642 - accuracy: 0.8567 - val_loss: 0.5934 - val_accuracy: 0.8233
Epoch 136/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4500 - accuracy: 0.8613 - val_loss: 0.5779 - val_accuracy: 0.8240
Epoch 137/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4342 - accuracy: 0.8617 - val_loss: 0.5920 - val_accuracy: 0.8273
Epoch 138/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4309 - accuracy: 0.8619 - val_loss: 0.5825 - val_accuracy: 0.8263
Epoch 139/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4113 - accuracy: 0.8692 - val_loss: 0.6172 - val_accuracy: 0.8153
Epoch 140/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4261 - accuracy: 0.8706 - val_loss: 0.5540 - val_accuracy: 0.8323
Epoch 141/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4161 - accuracy: 0.8721 - val_loss: 0.5357 - val_accuracy: 0.8357
Epoch 142/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4062 - accuracy: 0.8717 - val_loss: 0.5583 - val_accuracy: 0.8340
Epoch 143/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3960 - accuracy: 0.8774 - val_loss: 0.5832 - val_accuracy: 0.8263
Epoch 144/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3880 - accuracy: 0.8786 - val_loss: 0.5630 - val_accuracy: 0.8267
Epoch 145/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3900 - accuracy: 0.8779 - val_loss: 0.5420 - val_accuracy: 0.8390
Epoch 146/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3810 - accuracy: 0.8774 - val_loss: 0.5791 - val_accuracy: 0.8240
Epoch 147/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3765 - accuracy: 0.8830 - val_loss: 0.6042 - val_accuracy: 0.8190
Epoch 148/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3607 - accuracy: 0.8878 - val_loss: 0.6006 - val_accuracy: 0.8247
Epoch 149/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3574 - accuracy: 0.8888 - val_loss: 0.5257 - val_accuracy: 0.8427
Epoch 150/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3525 - accuracy: 0.8912 - val_loss: 0.5241 - val_accuracy: 0.8463
Epoch 151/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3520 - accuracy: 0.8893 - val_loss: 0.5115 - val_accuracy: 0.8480
Epoch 152/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3468 - accuracy: 0.8912 - val_loss: 0.5211 - val_accuracy: 0.8490
Epoch 153/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3385 - accuracy: 0.8973 - val_loss: 0.5484 - val_accuracy: 0.8417
Epoch 154/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3421 - accuracy: 0.8961 - val_loss: 0.5243 - val_accuracy: 0.8480
Epoch 155/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3210 - accuracy: 0.9001 - val_loss: 0.5056 - val_accuracy: 0.8500
Epoch 156/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3276 - accuracy: 0.8969 - val_loss: 0.5354 - val_accuracy: 0.8450
Epoch 157/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3224 - accuracy: 0.9022 - val_loss: 0.5145 - val_accuracy: 0.8523
Epoch 158/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3097 - accuracy: 0.9034 - val_loss: 0.6487 - val_accuracy: 0.8153
Epoch 159/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3119 - accuracy: 0.9019 - val_loss: 0.5268 - val_accuracy: 0.8437
Epoch 160/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3215 - accuracy: 0.8986 - val_loss: 0.5215 - val_accuracy: 0.8523
Epoch 161/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3034 - accuracy: 0.9043 - val_loss: 0.5214 - val_accuracy: 0.8540
Epoch 162/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2916 - accuracy: 0.9075 - val_loss: 0.5143 - val_accuracy: 0.8550
Epoch 163/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2950 - accuracy: 0.9062 - val_loss: 0.4717 - val_accuracy: 0.8617
Epoch 164/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2899 - accuracy: 0.9097 - val_loss: 0.4999 - val_accuracy: 0.8547
Epoch 165/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2711 - accuracy: 0.9145 - val_loss: 0.5128 - val_accuracy: 0.8520
Epoch 166/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2727 - accuracy: 0.9158 - val_loss: 0.5196 - val_accuracy: 0.8537
Epoch 167/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2786 - accuracy: 0.9142 - val_loss: 0.4806 - val_accuracy: 0.8590
Epoch 168/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2646 - accuracy: 0.9175 - val_loss: 0.4853 - val_accuracy: 0.8580
Epoch 169/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2600 - accuracy: 0.9214 - val_loss: 0.4936 - val_accuracy: 0.8607
Epoch 170/300
903/903 [==============================] - 7s 7ms/step - loss: 0.2601 - accuracy: 0.9222 - val_loss: 0.5299 - val_accuracy: 0.8503
Epoch 171/300
903/903 [==============================] - 8s 8ms/step - loss: 0.2591 - accuracy: 0.9204 - val_loss: 0.4719 - val_accuracy: 0.8660
Epoch 172/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2381 - accuracy: 0.9257 - val_loss: 0.5448 - val_accuracy: 0.8543
Epoch 173/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2394 - accuracy: 0.9270 - val_loss: 0.4850 - val_accuracy: 0.8633
Epoch 174/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2502 - accuracy: 0.9258 - val_loss: 0.4609 - val_accuracy: 0.8653
Epoch 175/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2461 - accuracy: 0.9250 - val_loss: 0.5050 - val_accuracy: 0.8593
Epoch 176/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2169 - accuracy: 0.9305 - val_loss: 0.5011 - val_accuracy: 0.8603
Epoch 177/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2311 - accuracy: 0.9304 - val_loss: 0.4999 - val_accuracy: 0.8623
Epoch 178/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2292 - accuracy: 0.9290 - val_loss: 0.4954 - val_accuracy: 0.8680
Epoch 179/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2310 - accuracy: 0.9311 - val_loss: 0.4877 - val_accuracy: 0.8623
Epoch 180/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2217 - accuracy: 0.9281 - val_loss: 0.4607 - val_accuracy: 0.8687
Epoch 181/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2069 - accuracy: 0.9377 - val_loss: 0.4993 - val_accuracy: 0.8560
Epoch 182/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2118 - accuracy: 0.9342 - val_loss: 0.4625 - val_accuracy: 0.8703
Epoch 183/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2025 - accuracy: 0.9399 - val_loss: 0.4946 - val_accuracy: 0.8623
Epoch 184/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2048 - accuracy: 0.9356 - val_loss: 0.4619 - val_accuracy: 0.8707
Epoch 185/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1996 - accuracy: 0.9410 - val_loss: 0.5295 - val_accuracy: 0.8570
Epoch 186/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2052 - accuracy: 0.9380 - val_loss: 0.4556 - val_accuracy: 0.8720
Epoch 187/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1987 - accuracy: 0.9416 - val_loss: 0.4781 - val_accuracy: 0.8757
Epoch 188/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1897 - accuracy: 0.9424 - val_loss: 0.4597 - val_accuracy: 0.8793
Epoch 189/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1895 - accuracy: 0.9414 - val_loss: 0.4871 - val_accuracy: 0.8663
Epoch 190/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1854 - accuracy: 0.9465 - val_loss: 0.4611 - val_accuracy: 0.8733
Epoch 191/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1852 - accuracy: 0.9423 - val_loss: 0.5088 - val_accuracy: 0.8600
Epoch 192/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1838 - accuracy: 0.9433 - val_loss: 0.4531 - val_accuracy: 0.8770
Epoch 193/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1827 - accuracy: 0.9444 - val_loss: 0.4908 - val_accuracy: 0.8733
Epoch 194/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1784 - accuracy: 0.9479 - val_loss: 0.4529 - val_accuracy: 0.8787
Epoch 195/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1676 - accuracy: 0.9521 - val_loss: 0.4769 - val_accuracy: 0.8743
Epoch 196/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1638 - accuracy: 0.9499 - val_loss: 0.4953 - val_accuracy: 0.8693
Epoch 197/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1645 - accuracy: 0.9488 - val_loss: 0.4831 - val_accuracy: 0.8660
Epoch 198/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1689 - accuracy: 0.9489 - val_loss: 0.4588 - val_accuracy: 0.8790
Epoch 199/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1511 - accuracy: 0.9575 - val_loss: 0.5095 - val_accuracy: 0.8713
Epoch 200/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1607 - accuracy: 0.9526 - val_loss: 0.4513 - val_accuracy: 0.8823
Epoch 201/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1453 - accuracy: 0.9546 - val_loss: 0.4856 - val_accuracy: 0.8800
Epoch 202/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1481 - accuracy: 0.9548 - val_loss: 0.4897 - val_accuracy: 0.8793
Epoch 203/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1438 - accuracy: 0.9578 - val_loss: 0.4787 - val_accuracy: 0.8770
Epoch 204/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1545 - accuracy: 0.9521 - val_loss: 0.4480 - val_accuracy: 0.8843
Epoch 205/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1420 - accuracy: 0.9559 - val_loss: 0.5031 - val_accuracy: 0.8727
Epoch 206/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1414 - accuracy: 0.9561 - val_loss: 0.4878 - val_accuracy: 0.8757
Epoch 207/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1419 - accuracy: 0.9571 - val_loss: 0.4838 - val_accuracy: 0.8817
Epoch 208/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1348 - accuracy: 0.9584 - val_loss: 0.4701 - val_accuracy: 0.8810
Epoch 209/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1390 - accuracy: 0.9588 - val_loss: 0.4721 - val_accuracy: 0.8807
Epoch 210/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1434 - accuracy: 0.9561 - val_loss: 0.4325 - val_accuracy: 0.8897
Epoch 211/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1211 - accuracy: 0.9642 - val_loss: 0.4953 - val_accuracy: 0.8790
Epoch 212/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1272 - accuracy: 0.9612 - val_loss: 0.4830 - val_accuracy: 0.8820
Epoch 213/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1368 - accuracy: 0.9596 - val_loss: 0.4742 - val_accuracy: 0.8827
Epoch 214/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1332 - accuracy: 0.9591 - val_loss: 0.4779 - val_accuracy: 0.8833
Epoch 215/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1250 - accuracy: 0.9606 - val_loss: 0.4599 - val_accuracy: 0.8843
Epoch 216/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1199 - accuracy: 0.9636 - val_loss: 0.4354 - val_accuracy: 0.8853
Epoch 217/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1087 - accuracy: 0.9670 - val_loss: 0.5013 - val_accuracy: 0.8830
Epoch 218/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1113 - accuracy: 0.9674 - val_loss: 0.4703 - val_accuracy: 0.8867
Epoch 219/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1144 - accuracy: 0.9664 - val_loss: 0.5120 - val_accuracy: 0.8790
Epoch 220/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1320 - accuracy: 0.9593 - val_loss: 0.4453 - val_accuracy: 0.8847
Epoch 221/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1109 - accuracy: 0.9657 - val_loss: 0.4343 - val_accuracy: 0.8907
Epoch 222/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1158 - accuracy: 0.9653 - val_loss: 0.5101 - val_accuracy: 0.8843
Epoch 223/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1047 - accuracy: 0.9672 - val_loss: 0.4912 - val_accuracy: 0.8847
Epoch 224/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1052 - accuracy: 0.9695 - val_loss: 0.5112 - val_accuracy: 0.8790
Epoch 225/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1014 - accuracy: 0.9691 - val_loss: 0.4466 - val_accuracy: 0.8863
Epoch 226/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1053 - accuracy: 0.9672 - val_loss: 0.4797 - val_accuracy: 0.8830
Epoch 227/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0976 - accuracy: 0.9712 - val_loss: 0.4432 - val_accuracy: 0.8893
Epoch 228/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1136 - accuracy: 0.9648 - val_loss: 0.4768 - val_accuracy: 0.8880
Epoch 229/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1022 - accuracy: 0.9704 - val_loss: 0.4616 - val_accuracy: 0.8887
Epoch 230/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0979 - accuracy: 0.9718 - val_loss: 0.5088 - val_accuracy: 0.8833
Epoch 231/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0923 - accuracy: 0.9734 - val_loss: 0.4610 - val_accuracy: 0.8913
Epoch 232/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0953 - accuracy: 0.9699 - val_loss: 0.4629 - val_accuracy: 0.8943
Epoch 233/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0888 - accuracy: 0.9743 - val_loss: 0.5332 - val_accuracy: 0.8687
Epoch 234/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0984 - accuracy: 0.9689 - val_loss: 0.4605 - val_accuracy: 0.8913
Epoch 235/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0942 - accuracy: 0.9713 - val_loss: 0.4889 - val_accuracy: 0.8877
Epoch 236/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0852 - accuracy: 0.9736 - val_loss: 0.4375 - val_accuracy: 0.8963
Epoch 237/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0907 - accuracy: 0.9724 - val_loss: 0.5157 - val_accuracy: 0.8787
Epoch 238/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0855 - accuracy: 0.9759 - val_loss: 0.5039 - val_accuracy: 0.8890
Epoch 239/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0907 - accuracy: 0.9744 - val_loss: 0.5088 - val_accuracy: 0.8853
Epoch 240/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0872 - accuracy: 0.9740 - val_loss: 0.4569 - val_accuracy: 0.8933
Epoch 241/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0905 - accuracy: 0.9730 - val_loss: 0.4924 - val_accuracy: 0.8937
Epoch 242/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0826 - accuracy: 0.9766 - val_loss: 0.5168 - val_accuracy: 0.8827
Epoch 243/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0806 - accuracy: 0.9763 - val_loss: 0.4887 - val_accuracy: 0.8843
Epoch 244/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0833 - accuracy: 0.9749 - val_loss: 0.4563 - val_accuracy: 0.8907
Epoch 245/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0831 - accuracy: 0.9753 - val_loss: 0.4752 - val_accuracy: 0.8940
Epoch 246/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0768 - accuracy: 0.9782 - val_loss: 0.5410 - val_accuracy: 0.8803
Epoch 247/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0833 - accuracy: 0.9759 - val_loss: 0.5266 - val_accuracy: 0.8847
Epoch 248/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0703 - accuracy: 0.9803 - val_loss: 0.5061 - val_accuracy: 0.8910
Epoch 249/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0777 - accuracy: 0.9761 - val_loss: 0.4994 - val_accuracy: 0.8897
Epoch 250/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0767 - accuracy: 0.9773 - val_loss: 0.5278 - val_accuracy: 0.8840
Epoch 251/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0785 - accuracy: 0.9771 - val_loss: 0.4832 - val_accuracy: 0.8957
Epoch 252/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0778 - accuracy: 0.9753 - val_loss: 0.4463 - val_accuracy: 0.8997
Epoch 253/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0718 - accuracy: 0.9788 - val_loss: 0.5368 - val_accuracy: 0.8880
Epoch 254/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0718 - accuracy: 0.9797 - val_loss: 0.4843 - val_accuracy: 0.8860
Epoch 255/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0713 - accuracy: 0.9787 - val_loss: 0.4697 - val_accuracy: 0.9010
Epoch 256/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0702 - accuracy: 0.9803 - val_loss: 0.4847 - val_accuracy: 0.8907
Epoch 257/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0633 - accuracy: 0.9804 - val_loss: 0.5517 - val_accuracy: 0.8813
Epoch 258/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0637 - accuracy: 0.9816 - val_loss: 0.5650 - val_accuracy: 0.8800
Epoch 259/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0770 - accuracy: 0.9776 - val_loss: 0.5458 - val_accuracy: 0.8770
Epoch 260/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0716 - accuracy: 0.9797 - val_loss: 0.4364 - val_accuracy: 0.9017
Epoch 261/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0652 - accuracy: 0.9800 - val_loss: 0.4831 - val_accuracy: 0.8933
Epoch 262/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0648 - accuracy: 0.9818 - val_loss: 0.5513 - val_accuracy: 0.8793
Epoch 263/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0608 - accuracy: 0.9839 - val_loss: 0.4780 - val_accuracy: 0.8940
Epoch 264/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0689 - accuracy: 0.9795 - val_loss: 0.5448 - val_accuracy: 0.8780
Epoch 265/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0683 - accuracy: 0.9782 - val_loss: 0.4797 - val_accuracy: 0.8977
Epoch 266/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0591 - accuracy: 0.9815 - val_loss: 0.4920 - val_accuracy: 0.8963
Epoch 267/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0649 - accuracy: 0.9794 - val_loss: 0.4838 - val_accuracy: 0.8920
Epoch 268/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0532 - accuracy: 0.9843 - val_loss: 0.5296 - val_accuracy: 0.8893
Epoch 269/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0616 - accuracy: 0.9818 - val_loss: 0.4759 - val_accuracy: 0.9023
Epoch 270/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0619 - accuracy: 0.9822 - val_loss: 0.5115 - val_accuracy: 0.8927
Epoch 271/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0651 - accuracy: 0.9806 - val_loss: 0.5212 - val_accuracy: 0.8943
Epoch 272/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0542 - accuracy: 0.9848 - val_loss: 0.5053 - val_accuracy: 0.8977
Epoch 273/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0563 - accuracy: 0.9846 - val_loss: 0.4779 - val_accuracy: 0.9017
Epoch 274/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0572 - accuracy: 0.9827 - val_loss: 0.5399 - val_accuracy: 0.8853
Epoch 275/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0600 - accuracy: 0.9816 - val_loss: 0.4699 - val_accuracy: 0.9010
Epoch 276/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0537 - accuracy: 0.9848 - val_loss: 0.4812 - val_accuracy: 0.8950
Epoch 277/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0550 - accuracy: 0.9829 - val_loss: 0.4848 - val_accuracy: 0.8983
Epoch 278/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0580 - accuracy: 0.9856 - val_loss: 0.4777 - val_accuracy: 0.9020
Epoch 279/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0478 - accuracy: 0.9866 - val_loss: 0.4817 - val_accuracy: 0.8993
Epoch 280/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0574 - accuracy: 0.9854 - val_loss: 0.4794 - val_accuracy: 0.8983
Epoch 281/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0518 - accuracy: 0.9833 - val_loss: 0.5089 - val_accuracy: 0.8947
Epoch 282/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0574 - accuracy: 0.9827 - val_loss: 0.4963 - val_accuracy: 0.9010
Epoch 283/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0535 - accuracy: 0.9848 - val_loss: 0.5023 - val_accuracy: 0.8967
Epoch 284/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0490 - accuracy: 0.9863 - val_loss: 0.5129 - val_accuracy: 0.9003
Epoch 285/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0527 - accuracy: 0.9846 - val_loss: 0.4955 - val_accuracy: 0.8977
Epoch 286/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0477 - accuracy: 0.9859 - val_loss: 0.5318 - val_accuracy: 0.8940
Epoch 287/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0455 - accuracy: 0.9875 - val_loss: 0.5229 - val_accuracy: 0.8950
Epoch 288/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0500 - accuracy: 0.9856 - val_loss: 0.6761 - val_accuracy: 0.8703
Epoch 289/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0599 - accuracy: 0.9824 - val_loss: 0.4626 - val_accuracy: 0.9007
Epoch 290/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0516 - accuracy: 0.9855 - val_loss: 0.5245 - val_accuracy: 0.8963
Epoch 291/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0433 - accuracy: 0.9875 - val_loss: 0.5311 - val_accuracy: 0.8907
Epoch 292/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0430 - accuracy: 0.9885 - val_loss: 0.4744 - val_accuracy: 0.8987
Epoch 293/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0537 - accuracy: 0.9842 - val_loss: 0.5632 - val_accuracy: 0.8807
Epoch 294/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0500 - accuracy: 0.9859 - val_loss: 0.4759 - val_accuracy: 0.9020
Epoch 295/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0417 - accuracy: 0.9878 - val_loss: 0.5017 - val_accuracy: 0.8990
Epoch 296/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0539 - accuracy: 0.9857 - val_loss: 0.5198 - val_accuracy: 0.8920
Epoch 297/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0443 - accuracy: 0.9866 - val_loss: 0.5078 - val_accuracy: 0.8980
Epoch 298/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0346 - accuracy: 0.9907 - val_loss: 0.5242 - val_accuracy: 0.8997
Epoch 299/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0503 - accuracy: 0.9853 - val_loss: 0.5034 - val_accuracy: 0.8980
Epoch 300/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0428 - accuracy: 0.9878 - val_loss: 0.5168 - val_accuracy: 0.8977

Conv2D_128_V2.summary()

Model: "Conv2D_128_V2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalised_data (Sequential  (None, None, None, 1)    0         
 )                                                               
                                                                 
 conv2d (Conv2D)             (None, 32, 32, 64)        1664      
                                                                 
 max_pooling2d (MaxPooling2D  (None, 16, 16, 64)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 8, 8, 128)         204928    
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 4, 4, 128)        0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 4, 4, 256)         295168    
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 2, 2, 256)        0         
 2D)                                                             
                                                                 
 conv2d_3 (Conv2D)           (None, 2, 2, 512)         1180160   
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 1, 1, 512)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 512)               0         
                                                                 
 dropout (Dropout)           (None, 512)               0         
                                                                 
 dense (Dense)               (None, 512)               262656    
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_1 (Dense)             (None, 128)               65664     
                                                                 
 dropout_2 (Dropout)         (None, 128)               0         
                                                                 
 dense_2 (Dense)             (None, 15)                1935      
                                                                 
=================================================================
Total params: 2,012,175
Trainable params: 2,012,175
Non-trainable params: 0
_________________________________________________________________

Observations

• This model definitely has more trainable parameters than the previous model as the number of epochs has increase and the architecture is much deeper

plot_learning_curve(Conv2D_128_V2_history.history)

Observations

• This architecture shows that the validation loss only converges at around 0.5+ which is not as low as the training loss when the number of epochs increase.
• Validation accuracy also starts to converge around 0.85+, which is not as ideal as the training accuracy.
• Both the initial loss and accuracy were quite similar in the first few epochs but start to diverge after.

Conv2D_128_V2.evaluate(test_data_128.batch(10))

300/300 [==============================] - 1s 3ms/step - loss: 0.4606 - accuracy: 0.9070

[0.46055060625076294, 0.9070000052452087]

Observations

• The test loss and test accuracy are definitely way better than the previous model built.
• However, we can further improve this by using a larger kernel size at first to capture larger spatial features and increase the receptive field, allowing the model to consider more context when making predictions.
• We can also try to increase/widen the fully connected layers or simply increasing the number of units in the FC layer to allow the model capture more intricate rel;ationships and representations.

CNN Version 3 (128 x 128)

• Next attempt to build a model by increasing the kernel size, the first fully connected layer and the Dropout layer (Introduces noise during training to help netowork be more robust).
• This can allow the convolutional layer to capture more global information in the input and hopefully help the accuracy and loss to converge towards the training model.

# Try for 128 x 128 images
tf.keras.backend.clear_session()

Conv2D_128_V3 = Sequential(
    name = "Conv2D_128_V3",
    layers = [
        normalised_data,
        Conv2D(64, (7, 7),input_shape=(128,128,1), activation = "relu", padding='same', strides = (4,4)),
        MaxPooling2D((2, 2)),
        Conv2D(128, (5, 5), activation = "relu", padding='same', strides = (2, 2)),
        MaxPooling2D((2, 2)),
        Conv2D(256, (3,3), activation = "relu", padding='same', strides = (1, 1)),
        MaxPooling2D((2, 2)),
        Conv2D(512, (3,3), activation = "relu", padding='same', strides = (1, 1)),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.5),

        Dense(1028, activation='relu'),
        Dropout(0.6),
        Dense(128, activation='relu'),
        Dropout(0.6),

        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.00001)
Conv2D_128_V3.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_128_V3.build(input_shape=(None, 128, 128, 1))

Conv2D_128_V3_history = Conv2D_128_V3.fit(
    train_data_128.batch(10), 
    epochs=300,
    validation_data=val_data_128.batch(10)
    )

Epoch 1/300
903/903 [==============================] - 6s 6ms/step - loss: 2.6890 - accuracy: 0.0912 - val_loss: 2.7191 - val_accuracy: 0.0667
Epoch 2/300
903/903 [==============================] - 6s 6ms/step - loss: 2.6674 - accuracy: 0.0967 - val_loss: 2.7212 - val_accuracy: 0.0710
Epoch 3/300
903/903 [==============================] - 6s 6ms/step - loss: 2.6575 - accuracy: 0.0937 - val_loss: 2.7188 - val_accuracy: 0.0903
Epoch 4/300
903/903 [==============================] - 6s 6ms/step - loss: 2.6458 - accuracy: 0.0939 - val_loss: 2.7195 - val_accuracy: 0.0963
Epoch 5/300
903/903 [==============================] - 6s 7ms/step - loss: 2.6353 - accuracy: 0.0943 - val_loss: 2.6966 - val_accuracy: 0.1037
Epoch 6/300
903/903 [==============================] - 6s 7ms/step - loss: 2.6121 - accuracy: 0.1084 - val_loss: 2.6572 - val_accuracy: 0.1670
Epoch 7/300
903/903 [==============================] - 6s 7ms/step - loss: 2.5854 - accuracy: 0.1156 - val_loss: 2.6086 - val_accuracy: 0.1193
Epoch 8/300
903/903 [==============================] - 6s 7ms/step - loss: 2.5507 - accuracy: 0.1256 - val_loss: 2.5683 - val_accuracy: 0.1283
Epoch 9/300
903/903 [==============================] - 6s 6ms/step - loss: 2.5202 - accuracy: 0.1426 - val_loss: 2.5513 - val_accuracy: 0.1313
Epoch 10/300
903/903 [==============================] - 6s 6ms/step - loss: 2.4971 - accuracy: 0.1491 - val_loss: 2.5103 - val_accuracy: 0.1497
Epoch 11/300
903/903 [==============================] - 6s 6ms/step - loss: 2.4728 - accuracy: 0.1562 - val_loss: 2.4907 - val_accuracy: 0.1510
Epoch 12/300
903/903 [==============================] - 6s 6ms/step - loss: 2.4552 - accuracy: 0.1655 - val_loss: 2.4692 - val_accuracy: 0.1527
Epoch 13/300
903/903 [==============================] - 6s 6ms/step - loss: 2.4455 - accuracy: 0.1652 - val_loss: 2.4443 - val_accuracy: 0.1570
Epoch 14/300
903/903 [==============================] - 6s 6ms/step - loss: 2.4176 - accuracy: 0.1720 - val_loss: 2.4217 - val_accuracy: 0.1653
Epoch 15/300
903/903 [==============================] - 6s 7ms/step - loss: 2.3972 - accuracy: 0.1830 - val_loss: 2.4053 - val_accuracy: 0.1930
Epoch 16/300
903/903 [==============================] - 6s 7ms/step - loss: 2.3762 - accuracy: 0.1865 - val_loss: 2.3757 - val_accuracy: 0.2200
Epoch 17/300
903/903 [==============================] - 6s 7ms/step - loss: 2.3446 - accuracy: 0.2013 - val_loss: 2.3460 - val_accuracy: 0.2457
Epoch 18/300
903/903 [==============================] - 6s 7ms/step - loss: 2.3199 - accuracy: 0.2134 - val_loss: 2.3276 - val_accuracy: 0.2723
Epoch 19/300
903/903 [==============================] - 6s 6ms/step - loss: 2.2869 - accuracy: 0.2201 - val_loss: 2.3184 - val_accuracy: 0.2690
Epoch 20/300
903/903 [==============================] - 6s 6ms/step - loss: 2.2629 - accuracy: 0.2404 - val_loss: 2.2652 - val_accuracy: 0.2823
Epoch 21/300
903/903 [==============================] - 6s 6ms/step - loss: 2.2260 - accuracy: 0.2538 - val_loss: 2.2577 - val_accuracy: 0.2570
Epoch 22/300
903/903 [==============================] - 6s 6ms/step - loss: 2.1978 - accuracy: 0.2654 - val_loss: 2.1952 - val_accuracy: 0.3037
Epoch 23/300
903/903 [==============================] - 6s 6ms/step - loss: 2.1733 - accuracy: 0.2782 - val_loss: 2.1636 - val_accuracy: 0.3177
Epoch 24/300
903/903 [==============================] - 6s 6ms/step - loss: 2.1446 - accuracy: 0.2871 - val_loss: 2.1324 - val_accuracy: 0.3273
Epoch 25/300
903/903 [==============================] - 6s 6ms/step - loss: 2.1272 - accuracy: 0.2965 - val_loss: 2.1180 - val_accuracy: 0.3217
Epoch 26/300
903/903 [==============================] - 6s 7ms/step - loss: 2.0990 - accuracy: 0.3017 - val_loss: 2.0840 - val_accuracy: 0.3290
Epoch 27/300
903/903 [==============================] - 6s 7ms/step - loss: 2.0691 - accuracy: 0.3110 - val_loss: 2.0465 - val_accuracy: 0.3507
Epoch 28/300
903/903 [==============================] - 6s 6ms/step - loss: 2.0384 - accuracy: 0.3232 - val_loss: 2.0846 - val_accuracy: 0.3227
Epoch 29/300
903/903 [==============================] - 6s 6ms/step - loss: 2.0154 - accuracy: 0.3322 - val_loss: 1.9930 - val_accuracy: 0.3600
Epoch 30/300
903/903 [==============================] - 6s 6ms/step - loss: 2.0125 - accuracy: 0.3326 - val_loss: 1.9991 - val_accuracy: 0.3487
Epoch 31/300
903/903 [==============================] - 6s 7ms/step - loss: 1.9795 - accuracy: 0.3428 - val_loss: 1.9677 - val_accuracy: 0.3590
Epoch 32/300
903/903 [==============================] - 6s 7ms/step - loss: 1.9485 - accuracy: 0.3603 - val_loss: 1.9210 - val_accuracy: 0.3763
Epoch 33/300
903/903 [==============================] - 6s 7ms/step - loss: 1.9292 - accuracy: 0.3640 - val_loss: 1.9071 - val_accuracy: 0.3767
Epoch 34/300
903/903 [==============================] - 6s 7ms/step - loss: 1.9062 - accuracy: 0.3696 - val_loss: 1.8651 - val_accuracy: 0.3893
Epoch 35/300
903/903 [==============================] - 6s 7ms/step - loss: 1.8917 - accuracy: 0.3799 - val_loss: 1.9024 - val_accuracy: 0.3733
Epoch 36/300
903/903 [==============================] - 6s 6ms/step - loss: 1.8629 - accuracy: 0.3869 - val_loss: 1.8146 - val_accuracy: 0.4030
Epoch 37/300
903/903 [==============================] - 6s 7ms/step - loss: 1.8428 - accuracy: 0.3890 - val_loss: 1.7976 - val_accuracy: 0.4027
Epoch 38/300
903/903 [==============================] - 6s 6ms/step - loss: 1.8266 - accuracy: 0.3918 - val_loss: 1.8022 - val_accuracy: 0.4133
Epoch 39/300
903/903 [==============================] - 6s 7ms/step - loss: 1.8008 - accuracy: 0.4037 - val_loss: 1.7624 - val_accuracy: 0.4170
Epoch 40/300
903/903 [==============================] - 6s 7ms/step - loss: 1.7807 - accuracy: 0.4125 - val_loss: 1.7351 - val_accuracy: 0.4280
Epoch 41/300
903/903 [==============================] - 6s 6ms/step - loss: 1.7619 - accuracy: 0.4168 - val_loss: 1.7503 - val_accuracy: 0.4193
Epoch 42/300
903/903 [==============================] - 6s 6ms/step - loss: 1.7388 - accuracy: 0.4271 - val_loss: 1.7126 - val_accuracy: 0.4277
Epoch 43/300
903/903 [==============================] - 6s 6ms/step - loss: 1.7148 - accuracy: 0.4319 - val_loss: 1.7659 - val_accuracy: 0.4073
Epoch 44/300
903/903 [==============================] - 6s 6ms/step - loss: 1.6846 - accuracy: 0.4432 - val_loss: 1.6543 - val_accuracy: 0.4473
Epoch 45/300
903/903 [==============================] - 6s 6ms/step - loss: 1.6664 - accuracy: 0.4485 - val_loss: 1.6285 - val_accuracy: 0.4523
Epoch 46/300
903/903 [==============================] - 6s 6ms/step - loss: 1.6562 - accuracy: 0.4541 - val_loss: 1.6223 - val_accuracy: 0.4590
Epoch 47/300
903/903 [==============================] - 6s 6ms/step - loss: 1.6339 - accuracy: 0.4599 - val_loss: 1.6157 - val_accuracy: 0.4610
Epoch 48/300
903/903 [==============================] - 6s 7ms/step - loss: 1.6132 - accuracy: 0.4677 - val_loss: 1.5714 - val_accuracy: 0.4727
Epoch 49/300
903/903 [==============================] - 6s 6ms/step - loss: 1.5951 - accuracy: 0.4680 - val_loss: 1.6153 - val_accuracy: 0.4703
Epoch 50/300
903/903 [==============================] - 6s 6ms/step - loss: 1.5746 - accuracy: 0.4797 - val_loss: 1.5466 - val_accuracy: 0.4780
Epoch 51/300
903/903 [==============================] - 6s 6ms/step - loss: 1.5582 - accuracy: 0.4901 - val_loss: 1.5234 - val_accuracy: 0.4877
Epoch 52/300
903/903 [==============================] - 6s 7ms/step - loss: 1.5257 - accuracy: 0.5032 - val_loss: 1.4794 - val_accuracy: 0.5030
Epoch 53/300
903/903 [==============================] - 6s 6ms/step - loss: 1.5178 - accuracy: 0.5022 - val_loss: 1.5498 - val_accuracy: 0.4830
Epoch 54/300
903/903 [==============================] - 6s 7ms/step - loss: 1.4885 - accuracy: 0.5156 - val_loss: 1.4425 - val_accuracy: 0.5230
Epoch 55/300
903/903 [==============================] - 6s 7ms/step - loss: 1.4690 - accuracy: 0.5152 - val_loss: 1.4344 - val_accuracy: 0.5300
Epoch 56/300
903/903 [==============================] - 6s 6ms/step - loss: 1.4411 - accuracy: 0.5223 - val_loss: 1.4126 - val_accuracy: 0.5317
Epoch 57/300
903/903 [==============================] - 6s 6ms/step - loss: 1.4306 - accuracy: 0.5335 - val_loss: 1.4008 - val_accuracy: 0.5360
Epoch 58/300
903/903 [==============================] - 6s 6ms/step - loss: 1.4153 - accuracy: 0.5382 - val_loss: 1.3764 - val_accuracy: 0.5567
Epoch 59/300
903/903 [==============================] - 6s 6ms/step - loss: 1.3925 - accuracy: 0.5439 - val_loss: 1.3591 - val_accuracy: 0.5640
Epoch 60/300
903/903 [==============================] - 6s 6ms/step - loss: 1.3703 - accuracy: 0.5519 - val_loss: 1.3795 - val_accuracy: 0.5370
Epoch 61/300
903/903 [==============================] - 6s 6ms/step - loss: 1.3487 - accuracy: 0.5603 - val_loss: 1.3292 - val_accuracy: 0.5727
Epoch 62/300
903/903 [==============================] - 6s 6ms/step - loss: 1.3388 - accuracy: 0.5601 - val_loss: 1.3144 - val_accuracy: 0.5677
Epoch 63/300
903/903 [==============================] - 6s 7ms/step - loss: 1.3151 - accuracy: 0.5717 - val_loss: 1.2847 - val_accuracy: 0.5800
Epoch 64/300
903/903 [==============================] - 6s 7ms/step - loss: 1.3021 - accuracy: 0.5732 - val_loss: 1.2719 - val_accuracy: 0.5820
Epoch 65/300
903/903 [==============================] - 6s 7ms/step - loss: 1.2729 - accuracy: 0.5835 - val_loss: 1.2747 - val_accuracy: 0.5767
Epoch 66/300
903/903 [==============================] - 6s 7ms/step - loss: 1.2532 - accuracy: 0.5892 - val_loss: 1.2494 - val_accuracy: 0.5853
Epoch 67/300
903/903 [==============================] - 6s 6ms/step - loss: 1.2435 - accuracy: 0.5943 - val_loss: 1.2713 - val_accuracy: 0.5927
Epoch 68/300
903/903 [==============================] - 6s 6ms/step - loss: 1.2305 - accuracy: 0.6011 - val_loss: 1.2203 - val_accuracy: 0.5967
Epoch 69/300
903/903 [==============================] - 6s 6ms/step - loss: 1.1942 - accuracy: 0.6156 - val_loss: 1.2201 - val_accuracy: 0.6037
Epoch 70/300
903/903 [==============================] - 6s 6ms/step - loss: 1.1738 - accuracy: 0.6196 - val_loss: 1.1680 - val_accuracy: 0.6110
Epoch 71/300
903/903 [==============================] - 6s 7ms/step - loss: 1.1577 - accuracy: 0.6176 - val_loss: 1.1340 - val_accuracy: 0.6230
Epoch 72/300
903/903 [==============================] - 6s 6ms/step - loss: 1.1502 - accuracy: 0.6316 - val_loss: 1.1481 - val_accuracy: 0.6183
Epoch 73/300
903/903 [==============================] - 6s 7ms/step - loss: 1.1290 - accuracy: 0.6323 - val_loss: 1.1984 - val_accuracy: 0.6070
Epoch 74/300
903/903 [==============================] - 6s 6ms/step - loss: 1.1148 - accuracy: 0.6368 - val_loss: 1.1492 - val_accuracy: 0.6177
Epoch 75/300
903/903 [==============================] - 6s 6ms/step - loss: 1.0961 - accuracy: 0.6465 - val_loss: 1.1144 - val_accuracy: 0.6290
Epoch 76/300
903/903 [==============================] - 6s 6ms/step - loss: 1.0930 - accuracy: 0.6463 - val_loss: 1.0922 - val_accuracy: 0.6367
Epoch 77/300
903/903 [==============================] - 6s 7ms/step - loss: 1.0652 - accuracy: 0.6588 - val_loss: 1.0846 - val_accuracy: 0.6363
Epoch 78/300
903/903 [==============================] - 6s 7ms/step - loss: 1.0544 - accuracy: 0.6608 - val_loss: 1.0681 - val_accuracy: 0.6423
Epoch 79/300
903/903 [==============================] - 6s 7ms/step - loss: 1.0334 - accuracy: 0.6711 - val_loss: 1.0444 - val_accuracy: 0.6507
Epoch 80/300
903/903 [==============================] - 6s 7ms/step - loss: 1.0261 - accuracy: 0.6711 - val_loss: 1.0343 - val_accuracy: 0.6533
Epoch 81/300
903/903 [==============================] - 6s 6ms/step - loss: 1.0103 - accuracy: 0.6698 - val_loss: 1.0154 - val_accuracy: 0.6643
Epoch 82/300
903/903 [==============================] - 6s 6ms/step - loss: 0.9765 - accuracy: 0.6883 - val_loss: 1.0037 - val_accuracy: 0.6690
Epoch 83/300
903/903 [==============================] - 6s 7ms/step - loss: 0.9735 - accuracy: 0.6856 - val_loss: 0.9940 - val_accuracy: 0.6720
Epoch 84/300
903/903 [==============================] - 6s 6ms/step - loss: 0.9603 - accuracy: 0.6895 - val_loss: 0.9856 - val_accuracy: 0.6743
Epoch 85/300
903/903 [==============================] - 6s 7ms/step - loss: 0.9502 - accuracy: 0.6956 - val_loss: 0.9579 - val_accuracy: 0.6923
Epoch 86/300
903/903 [==============================] - 6s 6ms/step - loss: 0.9245 - accuracy: 0.7025 - val_loss: 0.9741 - val_accuracy: 0.6760
Epoch 87/300
903/903 [==============================] - 6s 6ms/step - loss: 0.9316 - accuracy: 0.7025 - val_loss: 0.9630 - val_accuracy: 0.6967
Epoch 88/300
903/903 [==============================] - 6s 6ms/step - loss: 0.9002 - accuracy: 0.7072 - val_loss: 0.9337 - val_accuracy: 0.6950
Epoch 89/300
903/903 [==============================] - 6s 6ms/step - loss: 0.9047 - accuracy: 0.7099 - val_loss: 0.9203 - val_accuracy: 0.6957
Epoch 90/300
903/903 [==============================] - 6s 6ms/step - loss: 0.8828 - accuracy: 0.7212 - val_loss: 0.9134 - val_accuracy: 0.7123
Epoch 91/300
903/903 [==============================] - 6s 6ms/step - loss: 0.8791 - accuracy: 0.7173 - val_loss: 0.9235 - val_accuracy: 0.7093
Epoch 92/300
903/903 [==============================] - 6s 6ms/step - loss: 0.8534 - accuracy: 0.7307 - val_loss: 0.8777 - val_accuracy: 0.7183
Epoch 93/300
903/903 [==============================] - 6s 7ms/step - loss: 0.8325 - accuracy: 0.7370 - val_loss: 0.8750 - val_accuracy: 0.7267
Epoch 94/300
903/903 [==============================] - 6s 6ms/step - loss: 0.8419 - accuracy: 0.7339 - val_loss: 0.8635 - val_accuracy: 0.7263
Epoch 95/300
903/903 [==============================] - 6s 6ms/step - loss: 0.8308 - accuracy: 0.7348 - val_loss: 0.8452 - val_accuracy: 0.7270
Epoch 96/300
903/903 [==============================] - 6s 7ms/step - loss: 0.8049 - accuracy: 0.7472 - val_loss: 0.8924 - val_accuracy: 0.7103
Epoch 97/300
903/903 [==============================] - 6s 7ms/step - loss: 0.7883 - accuracy: 0.7513 - val_loss: 0.8587 - val_accuracy: 0.7327
Epoch 98/300
903/903 [==============================] - 6s 7ms/step - loss: 0.7813 - accuracy: 0.7553 - val_loss: 0.8159 - val_accuracy: 0.7367
Epoch 99/300
903/903 [==============================] - 6s 7ms/step - loss: 0.7698 - accuracy: 0.7565 - val_loss: 0.8228 - val_accuracy: 0.7400
Epoch 100/300
903/903 [==============================] - 6s 7ms/step - loss: 0.7601 - accuracy: 0.7540 - val_loss: 0.7877 - val_accuracy: 0.7573
Epoch 101/300
903/903 [==============================] - 6s 7ms/step - loss: 0.7534 - accuracy: 0.7560 - val_loss: 0.8219 - val_accuracy: 0.7407
Epoch 102/300
903/903 [==============================] - 6s 7ms/step - loss: 0.7416 - accuracy: 0.7650 - val_loss: 0.8266 - val_accuracy: 0.7377
Epoch 103/300
903/903 [==============================] - 6s 7ms/step - loss: 0.7210 - accuracy: 0.7750 - val_loss: 0.7847 - val_accuracy: 0.7590
Epoch 104/300
903/903 [==============================] - 6s 6ms/step - loss: 0.7228 - accuracy: 0.7724 - val_loss: 0.7607 - val_accuracy: 0.7640
Epoch 105/300
903/903 [==============================] - 6s 6ms/step - loss: 0.7029 - accuracy: 0.7812 - val_loss: 0.7615 - val_accuracy: 0.7600
Epoch 106/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6940 - accuracy: 0.7790 - val_loss: 0.7440 - val_accuracy: 0.7660
Epoch 107/300
903/903 [==============================] - 6s 7ms/step - loss: 0.6749 - accuracy: 0.7840 - val_loss: 0.7571 - val_accuracy: 0.7670
Epoch 108/300
903/903 [==============================] - 6s 7ms/step - loss: 0.6659 - accuracy: 0.7869 - val_loss: 0.7312 - val_accuracy: 0.7713
Epoch 109/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6643 - accuracy: 0.7904 - val_loss: 0.7401 - val_accuracy: 0.7687
Epoch 110/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6460 - accuracy: 0.7964 - val_loss: 0.6988 - val_accuracy: 0.7847
Epoch 111/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6397 - accuracy: 0.7988 - val_loss: 0.7227 - val_accuracy: 0.7803
Epoch 112/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6293 - accuracy: 0.8055 - val_loss: 0.7269 - val_accuracy: 0.7780
Epoch 113/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6142 - accuracy: 0.8041 - val_loss: 0.7030 - val_accuracy: 0.7777
Epoch 114/300
903/903 [==============================] - 6s 6ms/step - loss: 0.6153 - accuracy: 0.8051 - val_loss: 0.6968 - val_accuracy: 0.7850
Epoch 115/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5969 - accuracy: 0.8160 - val_loss: 0.6634 - val_accuracy: 0.7993
Epoch 116/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5870 - accuracy: 0.8171 - val_loss: 0.6752 - val_accuracy: 0.7967
Epoch 117/300
903/903 [==============================] - 6s 7ms/step - loss: 0.5828 - accuracy: 0.8161 - val_loss: 0.6733 - val_accuracy: 0.7993
Epoch 118/300
903/903 [==============================] - 6s 7ms/step - loss: 0.5537 - accuracy: 0.8258 - val_loss: 0.6454 - val_accuracy: 0.8107
Epoch 119/300
903/903 [==============================] - 6s 7ms/step - loss: 0.5578 - accuracy: 0.8278 - val_loss: 0.6276 - val_accuracy: 0.8073
Epoch 120/300
903/903 [==============================] - 6s 7ms/step - loss: 0.5672 - accuracy: 0.8258 - val_loss: 0.7371 - val_accuracy: 0.7690
Epoch 121/300
903/903 [==============================] - 6s 7ms/step - loss: 0.5539 - accuracy: 0.8271 - val_loss: 0.6698 - val_accuracy: 0.8000
Epoch 122/300
903/903 [==============================] - 6s 6ms/step - loss: 0.5433 - accuracy: 0.8303 - val_loss: 0.6294 - val_accuracy: 0.8087
Epoch 123/300
903/903 [==============================] - 6s 7ms/step - loss: 0.5306 - accuracy: 0.8401 - val_loss: 0.6837 - val_accuracy: 0.7937
Epoch 124/300
903/903 [==============================] - 6s 7ms/step - loss: 0.5343 - accuracy: 0.8314 - val_loss: 0.6105 - val_accuracy: 0.8183
Epoch 125/300
903/903 [==============================] - 6s 7ms/step - loss: 0.5211 - accuracy: 0.8374 - val_loss: 0.6773 - val_accuracy: 0.8010
Epoch 126/300
903/903 [==============================] - 6s 7ms/step - loss: 0.5005 - accuracy: 0.8447 - val_loss: 0.6011 - val_accuracy: 0.8230
Epoch 127/300
903/903 [==============================] - 6s 7ms/step - loss: 0.4981 - accuracy: 0.8412 - val_loss: 0.6387 - val_accuracy: 0.8073
Epoch 128/300
903/903 [==============================] - 6s 7ms/step - loss: 0.5002 - accuracy: 0.8463 - val_loss: 0.6382 - val_accuracy: 0.8140
Epoch 129/300
903/903 [==============================] - 6s 7ms/step - loss: 0.4785 - accuracy: 0.8511 - val_loss: 0.6720 - val_accuracy: 0.8020
Epoch 130/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4682 - accuracy: 0.8581 - val_loss: 0.6276 - val_accuracy: 0.8153
Epoch 131/300
903/903 [==============================] - 6s 7ms/step - loss: 0.4577 - accuracy: 0.8594 - val_loss: 0.5932 - val_accuracy: 0.8237
Epoch 132/300
903/903 [==============================] - 6s 7ms/step - loss: 0.4575 - accuracy: 0.8563 - val_loss: 0.6116 - val_accuracy: 0.8197
Epoch 133/300
903/903 [==============================] - 6s 7ms/step - loss: 0.4490 - accuracy: 0.8609 - val_loss: 0.6339 - val_accuracy: 0.8077
Epoch 134/300
903/903 [==============================] - 6s 6ms/step - loss: 0.4387 - accuracy: 0.8635 - val_loss: 0.6177 - val_accuracy: 0.8187
Epoch 135/300
903/903 [==============================] - 6s 7ms/step - loss: 0.4354 - accuracy: 0.8660 - val_loss: 0.5746 - val_accuracy: 0.8277
Epoch 136/300
903/903 [==============================] - 6s 7ms/step - loss: 0.4309 - accuracy: 0.8659 - val_loss: 0.5796 - val_accuracy: 0.8280
Epoch 137/300
903/903 [==============================] - 6s 7ms/step - loss: 0.4246 - accuracy: 0.8662 - val_loss: 0.6005 - val_accuracy: 0.8240
Epoch 138/300
903/903 [==============================] - 6s 7ms/step - loss: 0.4165 - accuracy: 0.8731 - val_loss: 0.6601 - val_accuracy: 0.7957
Epoch 139/300
903/903 [==============================] - 6s 7ms/step - loss: 0.4090 - accuracy: 0.8690 - val_loss: 0.5786 - val_accuracy: 0.8283
Epoch 140/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3928 - accuracy: 0.8813 - val_loss: 0.5560 - val_accuracy: 0.8363
Epoch 141/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3968 - accuracy: 0.8782 - val_loss: 0.5816 - val_accuracy: 0.8263
Epoch 142/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3964 - accuracy: 0.8755 - val_loss: 0.5454 - val_accuracy: 0.8437
Epoch 143/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3735 - accuracy: 0.8846 - val_loss: 0.5219 - val_accuracy: 0.8507
Epoch 144/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3766 - accuracy: 0.8814 - val_loss: 0.5666 - val_accuracy: 0.8373
Epoch 145/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3723 - accuracy: 0.8829 - val_loss: 0.5547 - val_accuracy: 0.8413
Epoch 146/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3620 - accuracy: 0.8911 - val_loss: 0.5583 - val_accuracy: 0.8413
Epoch 147/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3469 - accuracy: 0.8961 - val_loss: 0.5104 - val_accuracy: 0.8563
Epoch 148/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3508 - accuracy: 0.8900 - val_loss: 0.5669 - val_accuracy: 0.8393
Epoch 149/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3312 - accuracy: 0.8983 - val_loss: 0.5137 - val_accuracy: 0.8573
Epoch 150/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3360 - accuracy: 0.8957 - val_loss: 0.5295 - val_accuracy: 0.8523
Epoch 151/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3364 - accuracy: 0.8960 - val_loss: 0.5103 - val_accuracy: 0.8567
Epoch 152/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3205 - accuracy: 0.9008 - val_loss: 0.5455 - val_accuracy: 0.8440
Epoch 153/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3016 - accuracy: 0.9058 - val_loss: 0.5310 - val_accuracy: 0.8580
Epoch 154/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3100 - accuracy: 0.9031 - val_loss: 0.5115 - val_accuracy: 0.8590
Epoch 155/300
903/903 [==============================] - 6s 7ms/step - loss: 0.3136 - accuracy: 0.9061 - val_loss: 0.4798 - val_accuracy: 0.8673
Epoch 156/300
903/903 [==============================] - 6s 6ms/step - loss: 0.3117 - accuracy: 0.9061 - val_loss: 0.5323 - val_accuracy: 0.8540
Epoch 157/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2984 - accuracy: 0.9108 - val_loss: 0.4815 - val_accuracy: 0.8657
Epoch 158/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2950 - accuracy: 0.9074 - val_loss: 0.5157 - val_accuracy: 0.8503
Epoch 159/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2921 - accuracy: 0.9119 - val_loss: 0.5110 - val_accuracy: 0.8607
Epoch 160/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2799 - accuracy: 0.9163 - val_loss: 0.5204 - val_accuracy: 0.8637
Epoch 161/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2795 - accuracy: 0.9127 - val_loss: 0.4856 - val_accuracy: 0.8660
Epoch 162/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2770 - accuracy: 0.9171 - val_loss: 0.5152 - val_accuracy: 0.8510
Epoch 163/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2772 - accuracy: 0.9132 - val_loss: 0.4866 - val_accuracy: 0.8647
Epoch 164/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2651 - accuracy: 0.9157 - val_loss: 0.4839 - val_accuracy: 0.8660
Epoch 165/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2678 - accuracy: 0.9169 - val_loss: 0.4658 - val_accuracy: 0.8703
Epoch 166/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2611 - accuracy: 0.9218 - val_loss: 0.4912 - val_accuracy: 0.8693
Epoch 167/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2433 - accuracy: 0.9261 - val_loss: 0.4687 - val_accuracy: 0.8750
Epoch 168/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2513 - accuracy: 0.9227 - val_loss: 0.4916 - val_accuracy: 0.8687
Epoch 169/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2522 - accuracy: 0.9200 - val_loss: 0.4603 - val_accuracy: 0.8797
Epoch 170/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2514 - accuracy: 0.9241 - val_loss: 0.4708 - val_accuracy: 0.8737
Epoch 171/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2304 - accuracy: 0.9346 - val_loss: 0.5285 - val_accuracy: 0.8583
Epoch 172/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2254 - accuracy: 0.9302 - val_loss: 0.5059 - val_accuracy: 0.8647
Epoch 173/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2234 - accuracy: 0.9330 - val_loss: 0.5169 - val_accuracy: 0.8693
Epoch 174/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2283 - accuracy: 0.9280 - val_loss: 0.5174 - val_accuracy: 0.8660
Epoch 175/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2181 - accuracy: 0.9343 - val_loss: 0.4567 - val_accuracy: 0.8837
Epoch 176/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2140 - accuracy: 0.9346 - val_loss: 0.4510 - val_accuracy: 0.8863
Epoch 177/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2123 - accuracy: 0.9369 - val_loss: 0.4582 - val_accuracy: 0.8823
Epoch 178/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2054 - accuracy: 0.9343 - val_loss: 0.4949 - val_accuracy: 0.8713
Epoch 179/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2043 - accuracy: 0.9356 - val_loss: 0.4918 - val_accuracy: 0.8737
Epoch 180/300
903/903 [==============================] - 6s 6ms/step - loss: 0.2006 - accuracy: 0.9385 - val_loss: 0.4504 - val_accuracy: 0.8807
Epoch 181/300
903/903 [==============================] - 6s 7ms/step - loss: 0.2016 - accuracy: 0.9402 - val_loss: 0.4739 - val_accuracy: 0.8720
Epoch 182/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1906 - accuracy: 0.9411 - val_loss: 0.4796 - val_accuracy: 0.8780
Epoch 183/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1853 - accuracy: 0.9420 - val_loss: 0.4791 - val_accuracy: 0.8823
Epoch 184/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1902 - accuracy: 0.9447 - val_loss: 0.4548 - val_accuracy: 0.8867
Epoch 185/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1862 - accuracy: 0.9454 - val_loss: 0.5095 - val_accuracy: 0.8707
Epoch 186/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1897 - accuracy: 0.9453 - val_loss: 0.4589 - val_accuracy: 0.8813
Epoch 187/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1834 - accuracy: 0.9456 - val_loss: 0.4708 - val_accuracy: 0.8823
Epoch 188/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1712 - accuracy: 0.9476 - val_loss: 0.4685 - val_accuracy: 0.8800
Epoch 189/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1730 - accuracy: 0.9494 - val_loss: 0.4785 - val_accuracy: 0.8830
Epoch 190/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1712 - accuracy: 0.9463 - val_loss: 0.4484 - val_accuracy: 0.8850
Epoch 191/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1730 - accuracy: 0.9453 - val_loss: 0.4837 - val_accuracy: 0.8837
Epoch 192/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1624 - accuracy: 0.9510 - val_loss: 0.4962 - val_accuracy: 0.8813
Epoch 193/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1688 - accuracy: 0.9495 - val_loss: 0.6254 - val_accuracy: 0.8513
Epoch 194/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1604 - accuracy: 0.9510 - val_loss: 0.4467 - val_accuracy: 0.8930
Epoch 195/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1556 - accuracy: 0.9538 - val_loss: 0.4500 - val_accuracy: 0.8903
Epoch 196/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1593 - accuracy: 0.9517 - val_loss: 0.4704 - val_accuracy: 0.8867
Epoch 197/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1593 - accuracy: 0.9506 - val_loss: 0.4413 - val_accuracy: 0.8867
Epoch 198/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1506 - accuracy: 0.9545 - val_loss: 0.4533 - val_accuracy: 0.8893
Epoch 199/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1530 - accuracy: 0.9546 - val_loss: 0.4735 - val_accuracy: 0.8863
Epoch 200/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1439 - accuracy: 0.9599 - val_loss: 0.4976 - val_accuracy: 0.8810
Epoch 201/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1424 - accuracy: 0.9592 - val_loss: 0.4955 - val_accuracy: 0.8807
Epoch 202/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1456 - accuracy: 0.9548 - val_loss: 0.4992 - val_accuracy: 0.8807
Epoch 203/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1451 - accuracy: 0.9577 - val_loss: 0.5200 - val_accuracy: 0.8780
Epoch 204/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1335 - accuracy: 0.9595 - val_loss: 0.4659 - val_accuracy: 0.8907
Epoch 205/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1286 - accuracy: 0.9615 - val_loss: 0.4461 - val_accuracy: 0.8983
Epoch 206/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1393 - accuracy: 0.9587 - val_loss: 0.4602 - val_accuracy: 0.8933
Epoch 207/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1331 - accuracy: 0.9576 - val_loss: 0.4532 - val_accuracy: 0.8917
Epoch 208/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1267 - accuracy: 0.9618 - val_loss: 0.4538 - val_accuracy: 0.8940
Epoch 209/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1250 - accuracy: 0.9640 - val_loss: 0.4859 - val_accuracy: 0.8863
Epoch 210/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1233 - accuracy: 0.9627 - val_loss: 0.4974 - val_accuracy: 0.8863
Epoch 211/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1205 - accuracy: 0.9650 - val_loss: 0.4957 - val_accuracy: 0.8820
Epoch 212/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1260 - accuracy: 0.9648 - val_loss: 0.4509 - val_accuracy: 0.8950
Epoch 213/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1138 - accuracy: 0.9673 - val_loss: 0.4786 - val_accuracy: 0.8907
Epoch 214/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1312 - accuracy: 0.9592 - val_loss: 0.5429 - val_accuracy: 0.8767
Epoch 215/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1201 - accuracy: 0.9656 - val_loss: 0.4627 - val_accuracy: 0.8957
Epoch 216/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1180 - accuracy: 0.9671 - val_loss: 0.5509 - val_accuracy: 0.8803
Epoch 217/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1047 - accuracy: 0.9690 - val_loss: 0.5097 - val_accuracy: 0.8873
Epoch 218/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1150 - accuracy: 0.9649 - val_loss: 0.4684 - val_accuracy: 0.8937
Epoch 219/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1081 - accuracy: 0.9680 - val_loss: 0.4872 - val_accuracy: 0.8933
Epoch 220/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1059 - accuracy: 0.9670 - val_loss: 0.4689 - val_accuracy: 0.8917
Epoch 221/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0923 - accuracy: 0.9743 - val_loss: 0.4527 - val_accuracy: 0.9017
Epoch 222/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1002 - accuracy: 0.9729 - val_loss: 0.5014 - val_accuracy: 0.8910
Epoch 223/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1087 - accuracy: 0.9659 - val_loss: 0.4887 - val_accuracy: 0.8903
Epoch 224/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1061 - accuracy: 0.9695 - val_loss: 0.4579 - val_accuracy: 0.8940
Epoch 225/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1064 - accuracy: 0.9685 - val_loss: 0.4701 - val_accuracy: 0.8983
Epoch 226/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1025 - accuracy: 0.9710 - val_loss: 0.5397 - val_accuracy: 0.8843
Epoch 227/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1038 - accuracy: 0.9708 - val_loss: 0.4906 - val_accuracy: 0.8940
Epoch 228/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1017 - accuracy: 0.9693 - val_loss: 0.5037 - val_accuracy: 0.8927
Epoch 229/300
903/903 [==============================] - 6s 6ms/step - loss: 0.1031 - accuracy: 0.9694 - val_loss: 0.4636 - val_accuracy: 0.8933
Epoch 230/300
903/903 [==============================] - 6s 7ms/step - loss: 0.1015 - accuracy: 0.9715 - val_loss: 0.4708 - val_accuracy: 0.8940
Epoch 231/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0840 - accuracy: 0.9760 - val_loss: 0.4658 - val_accuracy: 0.9017
Epoch 232/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0911 - accuracy: 0.9736 - val_loss: 0.4836 - val_accuracy: 0.8990
Epoch 233/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0799 - accuracy: 0.9759 - val_loss: 0.5432 - val_accuracy: 0.8847
Epoch 234/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0898 - accuracy: 0.9743 - val_loss: 0.5338 - val_accuracy: 0.8867
Epoch 235/300
903/903 [==============================] - 6s 6ms/step - loss: 0.0933 - accuracy: 0.9728 - val_loss: 0.6806 - val_accuracy: 0.8637
Epoch 236/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0775 - accuracy: 0.9785 - val_loss: 0.4975 - val_accuracy: 0.8973
Epoch 237/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0861 - accuracy: 0.9747 - val_loss: 0.5373 - val_accuracy: 0.8897
Epoch 238/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0841 - accuracy: 0.9750 - val_loss: 0.5109 - val_accuracy: 0.8970
Epoch 239/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0784 - accuracy: 0.9778 - val_loss: 0.4647 - val_accuracy: 0.9027
Epoch 240/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0861 - accuracy: 0.9759 - val_loss: 0.6021 - val_accuracy: 0.8797
Epoch 241/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0789 - accuracy: 0.9782 - val_loss: 0.5009 - val_accuracy: 0.9000
Epoch 242/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0856 - accuracy: 0.9743 - val_loss: 0.5100 - val_accuracy: 0.8997
Epoch 243/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0791 - accuracy: 0.9774 - val_loss: 0.4719 - val_accuracy: 0.9010
Epoch 244/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0723 - accuracy: 0.9804 - val_loss: 0.5342 - val_accuracy: 0.8887
Epoch 245/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0717 - accuracy: 0.9784 - val_loss: 0.5419 - val_accuracy: 0.8913
Epoch 246/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0783 - accuracy: 0.9766 - val_loss: 0.4968 - val_accuracy: 0.9000
Epoch 247/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0824 - accuracy: 0.9767 - val_loss: 0.4730 - val_accuracy: 0.9050
Epoch 248/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0724 - accuracy: 0.9792 - val_loss: 0.4719 - val_accuracy: 0.9037
Epoch 249/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0719 - accuracy: 0.9807 - val_loss: 0.4560 - val_accuracy: 0.9087
Epoch 250/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0743 - accuracy: 0.9792 - val_loss: 0.4764 - val_accuracy: 0.9017
Epoch 251/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0718 - accuracy: 0.9797 - val_loss: 0.6576 - val_accuracy: 0.8573
Epoch 252/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0720 - accuracy: 0.9801 - val_loss: 0.4834 - val_accuracy: 0.9027
Epoch 253/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0719 - accuracy: 0.9794 - val_loss: 0.4919 - val_accuracy: 0.9023
Epoch 254/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0807 - accuracy: 0.9770 - val_loss: 0.5514 - val_accuracy: 0.8847
Epoch 255/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0653 - accuracy: 0.9815 - val_loss: 0.4924 - val_accuracy: 0.8983
Epoch 256/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0714 - accuracy: 0.9823 - val_loss: 0.5028 - val_accuracy: 0.9043
Epoch 257/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0677 - accuracy: 0.9792 - val_loss: 0.4825 - val_accuracy: 0.8990
Epoch 258/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0612 - accuracy: 0.9815 - val_loss: 0.5219 - val_accuracy: 0.8980
Epoch 259/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0587 - accuracy: 0.9833 - val_loss: 0.5455 - val_accuracy: 0.8933
Epoch 260/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0622 - accuracy: 0.9818 - val_loss: 0.5571 - val_accuracy: 0.8977
Epoch 261/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0646 - accuracy: 0.9809 - val_loss: 0.4961 - val_accuracy: 0.9027
Epoch 262/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0601 - accuracy: 0.9826 - val_loss: 0.4622 - val_accuracy: 0.9063
Epoch 263/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0578 - accuracy: 0.9854 - val_loss: 0.4968 - val_accuracy: 0.9073
Epoch 264/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0634 - accuracy: 0.9816 - val_loss: 0.4935 - val_accuracy: 0.9093
Epoch 265/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0606 - accuracy: 0.9839 - val_loss: 0.4735 - val_accuracy: 0.9063
Epoch 266/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0613 - accuracy: 0.9832 - val_loss: 0.5067 - val_accuracy: 0.8983
Epoch 267/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0586 - accuracy: 0.9845 - val_loss: 0.4913 - val_accuracy: 0.9053
Epoch 268/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0631 - accuracy: 0.9838 - val_loss: 0.5096 - val_accuracy: 0.8987
Epoch 269/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0600 - accuracy: 0.9826 - val_loss: 0.5366 - val_accuracy: 0.8953
Epoch 270/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0570 - accuracy: 0.9849 - val_loss: 0.5108 - val_accuracy: 0.9033
Epoch 271/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0538 - accuracy: 0.9845 - val_loss: 0.4934 - val_accuracy: 0.9053
Epoch 272/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0574 - accuracy: 0.9844 - val_loss: 0.4875 - val_accuracy: 0.9060
Epoch 273/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0546 - accuracy: 0.9846 - val_loss: 0.4903 - val_accuracy: 0.9053
Epoch 274/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0528 - accuracy: 0.9835 - val_loss: 0.5116 - val_accuracy: 0.9087
Epoch 275/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0503 - accuracy: 0.9849 - val_loss: 0.4967 - val_accuracy: 0.9033
Epoch 276/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0576 - accuracy: 0.9838 - val_loss: 0.4853 - val_accuracy: 0.9097
Epoch 277/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0540 - accuracy: 0.9848 - val_loss: 0.5350 - val_accuracy: 0.9000
Epoch 278/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0477 - accuracy: 0.9868 - val_loss: 0.4536 - val_accuracy: 0.9100
Epoch 279/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0513 - accuracy: 0.9865 - val_loss: 0.4955 - val_accuracy: 0.9040
Epoch 280/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0497 - accuracy: 0.9858 - val_loss: 0.4944 - val_accuracy: 0.9057
Epoch 281/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0519 - accuracy: 0.9842 - val_loss: 0.5667 - val_accuracy: 0.8877
Epoch 282/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0548 - accuracy: 0.9839 - val_loss: 0.5665 - val_accuracy: 0.8913
Epoch 283/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0397 - accuracy: 0.9901 - val_loss: 0.4740 - val_accuracy: 0.9120
Epoch 284/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0611 - accuracy: 0.9809 - val_loss: 0.5170 - val_accuracy: 0.9043
Epoch 285/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0473 - accuracy: 0.9866 - val_loss: 0.5404 - val_accuracy: 0.9003
Epoch 286/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0509 - accuracy: 0.9855 - val_loss: 0.5011 - val_accuracy: 0.9080
Epoch 287/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0490 - accuracy: 0.9858 - val_loss: 0.5081 - val_accuracy: 0.9033
Epoch 288/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0476 - accuracy: 0.9862 - val_loss: 0.5038 - val_accuracy: 0.9080
Epoch 289/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0518 - accuracy: 0.9862 - val_loss: 0.4795 - val_accuracy: 0.9060
Epoch 290/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0328 - accuracy: 0.9927 - val_loss: 0.5301 - val_accuracy: 0.9040
Epoch 291/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0531 - accuracy: 0.9869 - val_loss: 0.4960 - val_accuracy: 0.9073
Epoch 292/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0473 - accuracy: 0.9876 - val_loss: 0.4695 - val_accuracy: 0.9127
Epoch 293/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0391 - accuracy: 0.9893 - val_loss: 0.5110 - val_accuracy: 0.9080
Epoch 294/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0480 - accuracy: 0.9859 - val_loss: 0.5179 - val_accuracy: 0.9063
Epoch 295/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0413 - accuracy: 0.9873 - val_loss: 0.5489 - val_accuracy: 0.8997
Epoch 296/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0448 - accuracy: 0.9872 - val_loss: 0.5394 - val_accuracy: 0.9010
Epoch 297/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0397 - accuracy: 0.9879 - val_loss: 0.6361 - val_accuracy: 0.8877
Epoch 298/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0455 - accuracy: 0.9862 - val_loss: 0.5353 - val_accuracy: 0.9030
Epoch 299/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0379 - accuracy: 0.9891 - val_loss: 0.5079 - val_accuracy: 0.9077
Epoch 300/300
903/903 [==============================] - 6s 7ms/step - loss: 0.0370 - accuracy: 0.9891 - val_loss: 0.5270 - val_accuracy: 0.9033

Observations

• We have increased the number of units in the fully connected network to 1028 for the first Dense layer and also increased the kernel size to (7,7)

Conv2D_128_V3.summary()

Model: "Conv2D_128_V3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalised_data (Sequential  (None, None, None, 1)    0         
 )                                                               
                                                                 
 conv2d (Conv2D)             (None, 32, 32, 64)        3200      
                                                                 
 max_pooling2d (MaxPooling2D  (None, 16, 16, 64)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 8, 8, 128)         204928    
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 4, 4, 128)        0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 4, 4, 256)         295168    
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 2, 2, 256)        0         
 2D)                                                             
                                                                 
 conv2d_3 (Conv2D)           (None, 2, 2, 512)         1180160   
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 1, 1, 512)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 512)               0         
                                                                 
 dropout (Dropout)           (None, 512)               0         
                                                                 
 dense (Dense)               (None, 1028)              527364    
                                                                 
 dropout_1 (Dropout)         (None, 1028)              0         
                                                                 
 dense_1 (Dense)             (None, 128)               131712    
                                                                 
 dropout_2 (Dropout)         (None, 128)               0         
                                                                 
 dense_2 (Dense)             (None, 15)                1935      
                                                                 
=================================================================
Total params: 2,344,467
Trainable params: 2,344,467
Non-trainable params: 0
_________________________________________________________________

plot_learning_curve(Conv2D_128_V3_history.history)

Observations

• For the training and validation loss curve, both were converging quite well with few to none fluctuations and the validation loss curve converge around 0.60+.
• For the training and validation accuracy curve, both were increasing quite well at a steady rate and validation accuracy curve converge around 0.9+.
• Generally, this architecture is well built but I will improve it to see how much better it can do.

Conv2D_128_V3.evaluate(test_data_128.batch(10))

300/300 [==============================] - 1s 3ms/step - loss: 0.5101 - accuracy: 0.9020

[0.5100677013397217, 0.9020000100135803]

Observations

• Test accuracy at 0.9 is quite high but we can improve this score.

31 x 31 images

CNN Version 1 (31 x 31)

• Since the image resolution is lower, I shall decrease the kernel size to capture more local field instead of the global field.
• I will also increase the fully connected layers units to allow the model to learn complex representations for a higher level of abstraction.

# Try for 31 x 31 images
tf.keras.backend.clear_session()

Conv2D_31 = Sequential(name="Conv2D_31V1",
    layers = [
        normalised_data,
        Conv2D(32, (3, 3),input_shape=(31, 31 ,1), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),
        Conv2D(64, (3, 3), activation='relu', padding='same'),
        Conv2D(128, (3, 3), activation='relu', padding='same'),
        Conv2D(128, (3, 3), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.5),

        Dense(512, activation='relu'),
        Dropout(0.5),
        Dense(256, activation='relu'),
        Dropout(0.5),
        Dense(256, activation='relu'),
        Dropout(0.5),

        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.00001)
Conv2D_31.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_31.build(input_shape=(None, 31, 31, 1))

Conv2D_31_history = Conv2D_31.fit(
    train_data_31.batch(10), 
    epochs=200,
    validation_data=val_data_31.batch(10)
    )

Epoch 1/200
903/903 [==============================] - 6s 6ms/step - loss: 2.6977 - accuracy: 0.0814 - val_loss: 2.7108 - val_accuracy: 0.0667
Epoch 2/200
903/903 [==============================] - 5s 5ms/step - loss: 2.6682 - accuracy: 0.0981 - val_loss: 2.7226 - val_accuracy: 0.0667
Epoch 3/200
903/903 [==============================] - 5s 5ms/step - loss: 2.6543 - accuracy: 0.1012 - val_loss: 2.7229 - val_accuracy: 0.0667
Epoch 4/200
903/903 [==============================] - 5s 5ms/step - loss: 2.6473 - accuracy: 0.1008 - val_loss: 2.7160 - val_accuracy: 0.0667
Epoch 5/200
903/903 [==============================] - 5s 5ms/step - loss: 2.6345 - accuracy: 0.1037 - val_loss: 2.7139 - val_accuracy: 0.0663
Epoch 6/200
903/903 [==============================] - 5s 5ms/step - loss: 2.6203 - accuracy: 0.1082 - val_loss: 2.6866 - val_accuracy: 0.0843
Epoch 7/200
903/903 [==============================] - 5s 5ms/step - loss: 2.5887 - accuracy: 0.1191 - val_loss: 2.6383 - val_accuracy: 0.1367
Epoch 8/200
903/903 [==============================] - 5s 5ms/step - loss: 2.5458 - accuracy: 0.1427 - val_loss: 2.5880 - val_accuracy: 0.1487
Epoch 9/200
903/903 [==============================] - 5s 5ms/step - loss: 2.5075 - accuracy: 0.1595 - val_loss: 2.5431 - val_accuracy: 0.1590
Epoch 10/200
903/903 [==============================] - 5s 5ms/step - loss: 2.4792 - accuracy: 0.1679 - val_loss: 2.5167 - val_accuracy: 0.1730
Epoch 11/200
903/903 [==============================] - 5s 5ms/step - loss: 2.4538 - accuracy: 0.1763 - val_loss: 2.4792 - val_accuracy: 0.1967
Epoch 12/200
903/903 [==============================] - 5s 5ms/step - loss: 2.4410 - accuracy: 0.1755 - val_loss: 2.4538 - val_accuracy: 0.2007
Epoch 13/200
903/903 [==============================] - 5s 5ms/step - loss: 2.4082 - accuracy: 0.1952 - val_loss: 2.4175 - val_accuracy: 0.2190
Epoch 14/200
903/903 [==============================] - 5s 5ms/step - loss: 2.3674 - accuracy: 0.2093 - val_loss: 2.3749 - val_accuracy: 0.2400
Epoch 15/200
903/903 [==============================] - 5s 5ms/step - loss: 2.3344 - accuracy: 0.2193 - val_loss: 2.3567 - val_accuracy: 0.2737
Epoch 16/200
903/903 [==============================] - 5s 5ms/step - loss: 2.3091 - accuracy: 0.2317 - val_loss: 2.2984 - val_accuracy: 0.2820
Epoch 17/200
903/903 [==============================] - 5s 5ms/step - loss: 2.2758 - accuracy: 0.2416 - val_loss: 2.2686 - val_accuracy: 0.2993
Epoch 18/200
903/903 [==============================] - 5s 5ms/step - loss: 2.2443 - accuracy: 0.2563 - val_loss: 2.2214 - val_accuracy: 0.3213
Epoch 19/200
903/903 [==============================] - 5s 5ms/step - loss: 2.2173 - accuracy: 0.2727 - val_loss: 2.1884 - val_accuracy: 0.3263
Epoch 20/200
903/903 [==============================] - 5s 5ms/step - loss: 2.1767 - accuracy: 0.2777 - val_loss: 2.1510 - val_accuracy: 0.3403
Epoch 21/200
903/903 [==============================] - 5s 5ms/step - loss: 2.1522 - accuracy: 0.2862 - val_loss: 2.1237 - val_accuracy: 0.3433
Epoch 22/200
903/903 [==============================] - 5s 5ms/step - loss: 2.1243 - accuracy: 0.2979 - val_loss: 2.1005 - val_accuracy: 0.3437
Epoch 23/200
903/903 [==============================] - 5s 5ms/step - loss: 2.0986 - accuracy: 0.2996 - val_loss: 2.0606 - val_accuracy: 0.3593
Epoch 24/200
903/903 [==============================] - 5s 5ms/step - loss: 2.0756 - accuracy: 0.3127 - val_loss: 2.0286 - val_accuracy: 0.3610
Epoch 25/200
903/903 [==============================] - 5s 5ms/step - loss: 2.0546 - accuracy: 0.3230 - val_loss: 2.0008 - val_accuracy: 0.3663
Epoch 26/200
903/903 [==============================] - 5s 5ms/step - loss: 2.0238 - accuracy: 0.3383 - val_loss: 1.9734 - val_accuracy: 0.3790
Epoch 27/200
903/903 [==============================] - 5s 5ms/step - loss: 2.0105 - accuracy: 0.3354 - val_loss: 1.9674 - val_accuracy: 0.3787
Epoch 28/200
903/903 [==============================] - 5s 5ms/step - loss: 1.9762 - accuracy: 0.3529 - val_loss: 1.9278 - val_accuracy: 0.3920
Epoch 29/200
903/903 [==============================] - 5s 5ms/step - loss: 1.9695 - accuracy: 0.3451 - val_loss: 1.8955 - val_accuracy: 0.4047
Epoch 30/200
903/903 [==============================] - 5s 5ms/step - loss: 1.9560 - accuracy: 0.3569 - val_loss: 1.8708 - val_accuracy: 0.4060
Epoch 31/200
903/903 [==============================] - 5s 5ms/step - loss: 1.9326 - accuracy: 0.3650 - val_loss: 1.8411 - val_accuracy: 0.4113
Epoch 32/200
903/903 [==============================] - 5s 5ms/step - loss: 1.9087 - accuracy: 0.3715 - val_loss: 1.8151 - val_accuracy: 0.4250
Epoch 33/200
903/903 [==============================] - 5s 5ms/step - loss: 1.8940 - accuracy: 0.3782 - val_loss: 1.8019 - val_accuracy: 0.4340
Epoch 34/200
903/903 [==============================] - 5s 5ms/step - loss: 1.8608 - accuracy: 0.3860 - val_loss: 1.7665 - val_accuracy: 0.4390
Epoch 35/200
903/903 [==============================] - 5s 5ms/step - loss: 1.8408 - accuracy: 0.3909 - val_loss: 1.7364 - val_accuracy: 0.4500
Epoch 36/200
903/903 [==============================] - 5s 5ms/step - loss: 1.8165 - accuracy: 0.4005 - val_loss: 1.7289 - val_accuracy: 0.4477
Epoch 37/200
903/903 [==============================] - 5s 5ms/step - loss: 1.8073 - accuracy: 0.4119 - val_loss: 1.7017 - val_accuracy: 0.4580
Epoch 38/200
903/903 [==============================] - 5s 5ms/step - loss: 1.7808 - accuracy: 0.4108 - val_loss: 1.6799 - val_accuracy: 0.4583
Epoch 39/200
903/903 [==============================] - 5s 5ms/step - loss: 1.7568 - accuracy: 0.4168 - val_loss: 1.6622 - val_accuracy: 0.4657
Epoch 40/200
903/903 [==============================] - 5s 5ms/step - loss: 1.7423 - accuracy: 0.4296 - val_loss: 1.6276 - val_accuracy: 0.4740
Epoch 41/200
903/903 [==============================] - 5s 5ms/step - loss: 1.7360 - accuracy: 0.4322 - val_loss: 1.6253 - val_accuracy: 0.4727
Epoch 42/200
903/903 [==============================] - 5s 5ms/step - loss: 1.7121 - accuracy: 0.4401 - val_loss: 1.6033 - val_accuracy: 0.4860
Epoch 43/200
903/903 [==============================] - 5s 5ms/step - loss: 1.6883 - accuracy: 0.4453 - val_loss: 1.5762 - val_accuracy: 0.4853
Epoch 44/200
903/903 [==============================] - 5s 5ms/step - loss: 1.6710 - accuracy: 0.4560 - val_loss: 1.5648 - val_accuracy: 0.4937
Epoch 45/200
903/903 [==============================] - 4s 5ms/step - loss: 1.6595 - accuracy: 0.4569 - val_loss: 1.5326 - val_accuracy: 0.4967
Epoch 46/200
903/903 [==============================] - 5s 5ms/step - loss: 1.6380 - accuracy: 0.4574 - val_loss: 1.5197 - val_accuracy: 0.4950
Epoch 47/200
903/903 [==============================] - 5s 5ms/step - loss: 1.6318 - accuracy: 0.4658 - val_loss: 1.5085 - val_accuracy: 0.5057
Epoch 48/200
903/903 [==============================] - 5s 5ms/step - loss: 1.6045 - accuracy: 0.4749 - val_loss: 1.5182 - val_accuracy: 0.5033
Epoch 49/200
903/903 [==============================] - 5s 5ms/step - loss: 1.5861 - accuracy: 0.4785 - val_loss: 1.4659 - val_accuracy: 0.5140
Epoch 50/200
903/903 [==============================] - 5s 5ms/step - loss: 1.5682 - accuracy: 0.4877 - val_loss: 1.4547 - val_accuracy: 0.5273
Epoch 51/200
903/903 [==============================] - 5s 5ms/step - loss: 1.5527 - accuracy: 0.4919 - val_loss: 1.4425 - val_accuracy: 0.5230
Epoch 52/200
903/903 [==============================] - 5s 5ms/step - loss: 1.5270 - accuracy: 0.4928 - val_loss: 1.4182 - val_accuracy: 0.5317
Epoch 53/200
903/903 [==============================] - 5s 5ms/step - loss: 1.5296 - accuracy: 0.5022 - val_loss: 1.4365 - val_accuracy: 0.5177
Epoch 54/200
903/903 [==============================] - 5s 5ms/step - loss: 1.5085 - accuracy: 0.5025 - val_loss: 1.3964 - val_accuracy: 0.5390
Epoch 55/200
903/903 [==============================] - 5s 5ms/step - loss: 1.4892 - accuracy: 0.5116 - val_loss: 1.3777 - val_accuracy: 0.5560
Epoch 56/200
903/903 [==============================] - 5s 5ms/step - loss: 1.4763 - accuracy: 0.5128 - val_loss: 1.3629 - val_accuracy: 0.5543
Epoch 57/200
903/903 [==============================] - 5s 5ms/step - loss: 1.4486 - accuracy: 0.5254 - val_loss: 1.3460 - val_accuracy: 0.5617
Epoch 58/200
903/903 [==============================] - 5s 5ms/step - loss: 1.4423 - accuracy: 0.5238 - val_loss: 1.3308 - val_accuracy: 0.5597
Epoch 59/200
903/903 [==============================] - 5s 5ms/step - loss: 1.4269 - accuracy: 0.5351 - val_loss: 1.3510 - val_accuracy: 0.5463
Epoch 60/200
903/903 [==============================] - 5s 5ms/step - loss: 1.4162 - accuracy: 0.5316 - val_loss: 1.2997 - val_accuracy: 0.5727
Epoch 61/200
903/903 [==============================] - 5s 5ms/step - loss: 1.3828 - accuracy: 0.5491 - val_loss: 1.3013 - val_accuracy: 0.5667
Epoch 62/200
903/903 [==============================] - 5s 5ms/step - loss: 1.3944 - accuracy: 0.5357 - val_loss: 1.2996 - val_accuracy: 0.5793
Epoch 63/200
903/903 [==============================] - 5s 5ms/step - loss: 1.3808 - accuracy: 0.5505 - val_loss: 1.2756 - val_accuracy: 0.5707
Epoch 64/200
903/903 [==============================] - 5s 5ms/step - loss: 1.3501 - accuracy: 0.5582 - val_loss: 1.2493 - val_accuracy: 0.5870
Epoch 65/200
903/903 [==============================] - 5s 5ms/step - loss: 1.3463 - accuracy: 0.5569 - val_loss: 1.2465 - val_accuracy: 0.5840
Epoch 66/200
903/903 [==============================] - 5s 5ms/step - loss: 1.3264 - accuracy: 0.5659 - val_loss: 1.2681 - val_accuracy: 0.5900
Epoch 67/200
903/903 [==============================] - 5s 5ms/step - loss: 1.3308 - accuracy: 0.5578 - val_loss: 1.2135 - val_accuracy: 0.6017
Epoch 68/200
903/903 [==============================] - 5s 5ms/step - loss: 1.3076 - accuracy: 0.5708 - val_loss: 1.2846 - val_accuracy: 0.5787
Epoch 69/200
903/903 [==============================] - 5s 5ms/step - loss: 1.2827 - accuracy: 0.5770 - val_loss: 1.1831 - val_accuracy: 0.6087
Epoch 70/200
903/903 [==============================] - 5s 5ms/step - loss: 1.2788 - accuracy: 0.5769 - val_loss: 1.1860 - val_accuracy: 0.6037
Epoch 71/200
903/903 [==============================] - 5s 5ms/step - loss: 1.2646 - accuracy: 0.5833 - val_loss: 1.1750 - val_accuracy: 0.6043
Epoch 72/200
903/903 [==============================] - 5s 5ms/step - loss: 1.2532 - accuracy: 0.5923 - val_loss: 1.1406 - val_accuracy: 0.6193
Epoch 73/200
903/903 [==============================] - 5s 5ms/step - loss: 1.2366 - accuracy: 0.5955 - val_loss: 1.1372 - val_accuracy: 0.6227
Epoch 74/200
903/903 [==============================] - 5s 5ms/step - loss: 1.2230 - accuracy: 0.6012 - val_loss: 1.1268 - val_accuracy: 0.6200
Epoch 75/200
903/903 [==============================] - 5s 5ms/step - loss: 1.2107 - accuracy: 0.6081 - val_loss: 1.1332 - val_accuracy: 0.6233
Epoch 76/200
903/903 [==============================] - 5s 5ms/step - loss: 1.2202 - accuracy: 0.6033 - val_loss: 1.1063 - val_accuracy: 0.6323
Epoch 77/200
903/903 [==============================] - 5s 5ms/step - loss: 1.1935 - accuracy: 0.6105 - val_loss: 1.0977 - val_accuracy: 0.6353
Epoch 78/200
903/903 [==============================] - 5s 5ms/step - loss: 1.1936 - accuracy: 0.6077 - val_loss: 1.1013 - val_accuracy: 0.6317
Epoch 79/200
903/903 [==============================] - 5s 5ms/step - loss: 1.1623 - accuracy: 0.6210 - val_loss: 1.0753 - val_accuracy: 0.6453
Epoch 80/200
903/903 [==============================] - 5s 5ms/step - loss: 1.1534 - accuracy: 0.6264 - val_loss: 1.0691 - val_accuracy: 0.6487
Epoch 81/200
903/903 [==============================] - 5s 5ms/step - loss: 1.1416 - accuracy: 0.6279 - val_loss: 1.0665 - val_accuracy: 0.6437
Epoch 82/200
903/903 [==============================] - 5s 5ms/step - loss: 1.1284 - accuracy: 0.6342 - val_loss: 1.1059 - val_accuracy: 0.6467
Epoch 83/200
903/903 [==============================] - 5s 5ms/step - loss: 1.1313 - accuracy: 0.6290 - val_loss: 1.1384 - val_accuracy: 0.6300
Epoch 84/200
903/903 [==============================] - 5s 5ms/step - loss: 1.1094 - accuracy: 0.6399 - val_loss: 1.0176 - val_accuracy: 0.6653
Epoch 85/200
903/903 [==============================] - 5s 5ms/step - loss: 1.1019 - accuracy: 0.6387 - val_loss: 1.0246 - val_accuracy: 0.6687
Epoch 86/200
903/903 [==============================] - 5s 5ms/step - loss: 1.0864 - accuracy: 0.6515 - val_loss: 0.9899 - val_accuracy: 0.6837
Epoch 87/200
903/903 [==============================] - 5s 5ms/step - loss: 1.0854 - accuracy: 0.6492 - val_loss: 0.9983 - val_accuracy: 0.6820
Epoch 88/200
903/903 [==============================] - 5s 5ms/step - loss: 1.0500 - accuracy: 0.6596 - val_loss: 0.9792 - val_accuracy: 0.6803
Epoch 89/200
903/903 [==============================] - 5s 5ms/step - loss: 1.0550 - accuracy: 0.6583 - val_loss: 0.9624 - val_accuracy: 0.6897
Epoch 90/200
903/903 [==============================] - 5s 5ms/step - loss: 1.0562 - accuracy: 0.6571 - val_loss: 1.0004 - val_accuracy: 0.6783
Epoch 91/200
903/903 [==============================] - 5s 5ms/step - loss: 1.0249 - accuracy: 0.6653 - val_loss: 0.9502 - val_accuracy: 0.6993
Epoch 92/200
903/903 [==============================] - 5s 5ms/step - loss: 1.0268 - accuracy: 0.6654 - val_loss: 0.9393 - val_accuracy: 0.6973
Epoch 93/200
903/903 [==============================] - 5s 5ms/step - loss: 1.0195 - accuracy: 0.6716 - val_loss: 0.9385 - val_accuracy: 0.7033
Epoch 94/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9961 - accuracy: 0.6828 - val_loss: 0.9479 - val_accuracy: 0.6997
Epoch 95/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9952 - accuracy: 0.6796 - val_loss: 0.9226 - val_accuracy: 0.7067
Epoch 96/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9852 - accuracy: 0.6819 - val_loss: 0.9188 - val_accuracy: 0.7033
Epoch 97/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9650 - accuracy: 0.6860 - val_loss: 0.8975 - val_accuracy: 0.7133
Epoch 98/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9605 - accuracy: 0.6902 - val_loss: 0.8948 - val_accuracy: 0.7160
Epoch 99/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9435 - accuracy: 0.6925 - val_loss: 0.8681 - val_accuracy: 0.7273
Epoch 100/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9488 - accuracy: 0.6938 - val_loss: 0.8678 - val_accuracy: 0.7263
Epoch 101/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9414 - accuracy: 0.7006 - val_loss: 0.8580 - val_accuracy: 0.7350
Epoch 102/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9153 - accuracy: 0.7049 - val_loss: 0.8691 - val_accuracy: 0.7287
Epoch 103/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9118 - accuracy: 0.7101 - val_loss: 0.8464 - val_accuracy: 0.7343
Epoch 104/200
903/903 [==============================] - 5s 5ms/step - loss: 0.9080 - accuracy: 0.7048 - val_loss: 0.8650 - val_accuracy: 0.7360
Epoch 105/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8913 - accuracy: 0.7164 - val_loss: 0.8277 - val_accuracy: 0.7447
Epoch 106/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8847 - accuracy: 0.7174 - val_loss: 0.8238 - val_accuracy: 0.7460
Epoch 107/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8676 - accuracy: 0.7204 - val_loss: 0.8050 - val_accuracy: 0.7533
Epoch 108/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8667 - accuracy: 0.7256 - val_loss: 0.7972 - val_accuracy: 0.7537
Epoch 109/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8748 - accuracy: 0.7193 - val_loss: 0.8045 - val_accuracy: 0.7517
Epoch 110/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8548 - accuracy: 0.7332 - val_loss: 0.7903 - val_accuracy: 0.7580
Epoch 111/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8440 - accuracy: 0.7245 - val_loss: 0.8007 - val_accuracy: 0.7550
Epoch 112/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8237 - accuracy: 0.7395 - val_loss: 0.7881 - val_accuracy: 0.7577
Epoch 113/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8279 - accuracy: 0.7368 - val_loss: 0.7746 - val_accuracy: 0.7610
Epoch 114/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8044 - accuracy: 0.7425 - val_loss: 0.7805 - val_accuracy: 0.7597
Epoch 115/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8138 - accuracy: 0.7375 - val_loss: 0.7657 - val_accuracy: 0.7663
Epoch 116/200
903/903 [==============================] - 5s 5ms/step - loss: 0.8139 - accuracy: 0.7414 - val_loss: 0.7685 - val_accuracy: 0.7607
Epoch 117/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7871 - accuracy: 0.7514 - val_loss: 0.7453 - val_accuracy: 0.7707
Epoch 118/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7805 - accuracy: 0.7497 - val_loss: 0.7424 - val_accuracy: 0.7663
Epoch 119/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7682 - accuracy: 0.7607 - val_loss: 0.7242 - val_accuracy: 0.7763
Epoch 120/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7704 - accuracy: 0.7563 - val_loss: 0.7182 - val_accuracy: 0.7833
Epoch 121/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7491 - accuracy: 0.7601 - val_loss: 0.7096 - val_accuracy: 0.7777
Epoch 122/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7441 - accuracy: 0.7581 - val_loss: 0.6975 - val_accuracy: 0.7840
Epoch 123/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7524 - accuracy: 0.7630 - val_loss: 0.6959 - val_accuracy: 0.7853
Epoch 124/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7431 - accuracy: 0.7641 - val_loss: 0.6950 - val_accuracy: 0.7877
Epoch 125/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7346 - accuracy: 0.7612 - val_loss: 0.6837 - val_accuracy: 0.7900
Epoch 126/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7183 - accuracy: 0.7691 - val_loss: 0.6904 - val_accuracy: 0.7877
Epoch 127/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7069 - accuracy: 0.7759 - val_loss: 0.6853 - val_accuracy: 0.7883
Epoch 128/200
903/903 [==============================] - 5s 5ms/step - loss: 0.7026 - accuracy: 0.7774 - val_loss: 0.6655 - val_accuracy: 0.7970
Epoch 129/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6987 - accuracy: 0.7749 - val_loss: 0.6675 - val_accuracy: 0.7940
Epoch 130/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6909 - accuracy: 0.7829 - val_loss: 0.6511 - val_accuracy: 0.7980
Epoch 131/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6859 - accuracy: 0.7811 - val_loss: 0.6754 - val_accuracy: 0.7907
Epoch 132/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6695 - accuracy: 0.7889 - val_loss: 0.6584 - val_accuracy: 0.8027
Epoch 133/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6637 - accuracy: 0.7888 - val_loss: 0.6355 - val_accuracy: 0.8053
Epoch 134/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6575 - accuracy: 0.7901 - val_loss: 0.6282 - val_accuracy: 0.8053
Epoch 135/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6550 - accuracy: 0.7930 - val_loss: 0.6289 - val_accuracy: 0.8020
Epoch 136/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6569 - accuracy: 0.7959 - val_loss: 0.6191 - val_accuracy: 0.8083
Epoch 137/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6355 - accuracy: 0.8001 - val_loss: 0.6103 - val_accuracy: 0.8070
Epoch 138/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6304 - accuracy: 0.8049 - val_loss: 0.6001 - val_accuracy: 0.8153
Epoch 139/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6385 - accuracy: 0.7994 - val_loss: 0.6183 - val_accuracy: 0.8110
Epoch 140/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6202 - accuracy: 0.8023 - val_loss: 0.6052 - val_accuracy: 0.8100
Epoch 141/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6300 - accuracy: 0.7986 - val_loss: 0.5946 - val_accuracy: 0.8140
Epoch 142/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6137 - accuracy: 0.8066 - val_loss: 0.5977 - val_accuracy: 0.8180
Epoch 143/200
903/903 [==============================] - 5s 5ms/step - loss: 0.6045 - accuracy: 0.8099 - val_loss: 0.5978 - val_accuracy: 0.8180
Epoch 144/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5879 - accuracy: 0.8134 - val_loss: 0.5933 - val_accuracy: 0.8177
Epoch 145/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5839 - accuracy: 0.8141 - val_loss: 0.5839 - val_accuracy: 0.8187
Epoch 146/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5905 - accuracy: 0.8147 - val_loss: 0.5761 - val_accuracy: 0.8237
Epoch 147/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5698 - accuracy: 0.8179 - val_loss: 0.5790 - val_accuracy: 0.8230
Epoch 148/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5674 - accuracy: 0.8217 - val_loss: 0.5683 - val_accuracy: 0.8257
Epoch 149/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5757 - accuracy: 0.8180 - val_loss: 0.5737 - val_accuracy: 0.8257
Epoch 150/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5567 - accuracy: 0.8209 - val_loss: 0.5533 - val_accuracy: 0.8300
Epoch 151/200
903/903 [==============================] - 5s 6ms/step - loss: 0.5463 - accuracy: 0.8261 - val_loss: 0.5592 - val_accuracy: 0.8317
Epoch 152/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5568 - accuracy: 0.8251 - val_loss: 0.5487 - val_accuracy: 0.8313
Epoch 153/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5489 - accuracy: 0.8234 - val_loss: 0.5504 - val_accuracy: 0.8310
Epoch 154/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5352 - accuracy: 0.8363 - val_loss: 0.5413 - val_accuracy: 0.8337
Epoch 155/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5317 - accuracy: 0.8322 - val_loss: 0.5308 - val_accuracy: 0.8397
Epoch 156/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5118 - accuracy: 0.8391 - val_loss: 0.5380 - val_accuracy: 0.8363
Epoch 157/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5148 - accuracy: 0.8376 - val_loss: 0.5202 - val_accuracy: 0.8370
Epoch 158/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5098 - accuracy: 0.8376 - val_loss: 0.5357 - val_accuracy: 0.8363
Epoch 159/200
903/903 [==============================] - 5s 5ms/step - loss: 0.5035 - accuracy: 0.8385 - val_loss: 0.5341 - val_accuracy: 0.8387
Epoch 160/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4971 - accuracy: 0.8436 - val_loss: 0.5220 - val_accuracy: 0.8387
Epoch 161/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4866 - accuracy: 0.8475 - val_loss: 0.5145 - val_accuracy: 0.8423
Epoch 162/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4927 - accuracy: 0.8438 - val_loss: 0.5239 - val_accuracy: 0.8397
Epoch 163/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4865 - accuracy: 0.8488 - val_loss: 0.5564 - val_accuracy: 0.8280
Epoch 164/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4819 - accuracy: 0.8467 - val_loss: 0.5138 - val_accuracy: 0.8477
Epoch 165/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4706 - accuracy: 0.8517 - val_loss: 0.5077 - val_accuracy: 0.8463
Epoch 166/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4684 - accuracy: 0.8515 - val_loss: 0.4973 - val_accuracy: 0.8440
Epoch 167/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4521 - accuracy: 0.8578 - val_loss: 0.4964 - val_accuracy: 0.8507
Epoch 168/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4680 - accuracy: 0.8495 - val_loss: 0.5008 - val_accuracy: 0.8530
Epoch 169/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4554 - accuracy: 0.8568 - val_loss: 0.5006 - val_accuracy: 0.8503
Epoch 170/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4549 - accuracy: 0.8563 - val_loss: 0.4776 - val_accuracy: 0.8540
Epoch 171/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4512 - accuracy: 0.8604 - val_loss: 0.4839 - val_accuracy: 0.8500
Epoch 172/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4407 - accuracy: 0.8584 - val_loss: 0.4798 - val_accuracy: 0.8570
Epoch 173/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4305 - accuracy: 0.8621 - val_loss: 0.4913 - val_accuracy: 0.8467
Epoch 174/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4525 - accuracy: 0.8550 - val_loss: 0.4782 - val_accuracy: 0.8560
Epoch 175/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4163 - accuracy: 0.8716 - val_loss: 0.4732 - val_accuracy: 0.8573
Epoch 176/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4272 - accuracy: 0.8625 - val_loss: 0.4946 - val_accuracy: 0.8503
Epoch 177/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4097 - accuracy: 0.8726 - val_loss: 0.4704 - val_accuracy: 0.8567
Epoch 178/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4186 - accuracy: 0.8670 - val_loss: 0.4900 - val_accuracy: 0.8547
Epoch 179/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4181 - accuracy: 0.8661 - val_loss: 0.4706 - val_accuracy: 0.8587
Epoch 180/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4054 - accuracy: 0.8685 - val_loss: 0.4585 - val_accuracy: 0.8613
Epoch 181/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3941 - accuracy: 0.8768 - val_loss: 0.4599 - val_accuracy: 0.8617
Epoch 182/200
903/903 [==============================] - 5s 5ms/step - loss: 0.4036 - accuracy: 0.8701 - val_loss: 0.4672 - val_accuracy: 0.8593
Epoch 183/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3797 - accuracy: 0.8796 - val_loss: 0.4672 - val_accuracy: 0.8617
Epoch 184/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3934 - accuracy: 0.8756 - val_loss: 0.4575 - val_accuracy: 0.8627
Epoch 185/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3920 - accuracy: 0.8702 - val_loss: 0.4599 - val_accuracy: 0.8597
Epoch 186/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3739 - accuracy: 0.8841 - val_loss: 0.4444 - val_accuracy: 0.8673
Epoch 187/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3693 - accuracy: 0.8819 - val_loss: 0.4490 - val_accuracy: 0.8663
Epoch 188/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3691 - accuracy: 0.8823 - val_loss: 0.4398 - val_accuracy: 0.8677
Epoch 189/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3595 - accuracy: 0.8876 - val_loss: 0.4550 - val_accuracy: 0.8657
Epoch 190/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3676 - accuracy: 0.8819 - val_loss: 0.4334 - val_accuracy: 0.8713
Epoch 191/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3551 - accuracy: 0.8880 - val_loss: 0.4261 - val_accuracy: 0.8673
Epoch 192/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3538 - accuracy: 0.8870 - val_loss: 0.4458 - val_accuracy: 0.8713
Epoch 193/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3422 - accuracy: 0.8926 - val_loss: 0.4415 - val_accuracy: 0.8690
Epoch 194/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3544 - accuracy: 0.8888 - val_loss: 0.4520 - val_accuracy: 0.8707
Epoch 195/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3453 - accuracy: 0.8938 - val_loss: 0.4276 - val_accuracy: 0.8770
Epoch 196/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3397 - accuracy: 0.8941 - val_loss: 0.4282 - val_accuracy: 0.8733
Epoch 197/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3351 - accuracy: 0.8932 - val_loss: 0.4475 - val_accuracy: 0.8670
Epoch 198/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3366 - accuracy: 0.8963 - val_loss: 0.4195 - val_accuracy: 0.8747
Epoch 199/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3331 - accuracy: 0.8958 - val_loss: 0.4173 - val_accuracy: 0.8760
Epoch 200/200
903/903 [==============================] - 5s 5ms/step - loss: 0.3229 - accuracy: 0.8962 - val_loss: 0.4178 - val_accuracy: 0.8797

Observations

• As you can see, for the model that deals with 31 x 31 images, I have added more layers for the fully connected layers as this will help the neural network greater expressiveness, enabling it to learn and represent more sophisticated features and relationship within the data.
• With a lower pixelated image like 31 x 31, I have to use smaller filters to allow the network to capture more local and fine-grained patterns in the input data, starting from 32 filters.

Conv2D_31.summary()

Model: "Conv2D_31V1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalised_data (Sequential  (None, None, None, 1)    0         
 )                                                               
                                                                 
 conv2d (Conv2D)             (None, 31, 31, 32)        320       
                                                                 
 max_pooling2d (MaxPooling2D  (None, 15, 15, 32)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 15, 15, 64)        18496     
                                                                 
 conv2d_2 (Conv2D)           (None, 15, 15, 128)       73856     
                                                                 
 conv2d_3 (Conv2D)           (None, 15, 15, 128)       147584    
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 7, 7, 128)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 6272)              0         
                                                                 
 dropout (Dropout)           (None, 6272)              0         
                                                                 
 dense (Dense)               (None, 512)               3211776   
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_1 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               65792     
                                                                 
 dropout_3 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 15)                3855      
                                                                 
=================================================================
Total params: 3,653,007
Trainable params: 3,653,007
Non-trainable params: 0
_________________________________________________________________

Observations

• There are more trainable parameters for the 31 x 31 images than the 128 x 128 which may mean the models can still learn more complex and intricate patterns from the training data.

plot_learning_curve(Conv2D_31_history.history)

Observations

• From the training and validation loss curve, we can see the validation curve converges quite well around 0.3+ but maybe changing the learning rate and adding regularisation can improve this.
• The validation accuracy is quite high, reaching 0.94, but I think we can improve this model.

Conv2D_31.evaluate(test_data_31.batch(10))

300/300 [==============================] - 1s 2ms/step - loss: 0.3666 - accuracy: 0.8957

[0.3666374981403351, 0.8956666588783264]

Observations:

• The test accuracy as expected reaches 0.94, showing that this model is quite good, just need a few improvements.

Training with Data Augumentation

• Using the same network architecture as before, we shall see if the augmentation of data being upsampled into the existing training data to see if augmentating the data and balancing the data helps the model

128 x 128 images

CNN Augmented Balance Version 1 (128 x 128)

# Try for 128 x 128 images
tf.keras.backend.clear_session()

Conv2D_128V1_aug = Sequential(name="Conv2D_128V1_Augmentation",
    layers = [
        normalised_data,
        Conv2D(64, (5, 5), activation='relu', padding='same', input_shape=(128, 128, 1), strides=(4, 4)),
        MaxPooling2D((2, 2)),
        Conv2D(128, (3, 3), activation='relu', padding='same', strides=(1, 1)),
        MaxPooling2D((2, 2)),
        Conv2D(128, (3, 3), activation='relu', padding='same', strides=(1, 1)),
        MaxPooling2D((2, 2)),
        Conv2D(256, (3, 3), activation='relu', padding='same', strides=(1, 1)),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.5),

        Dense(1024, activation='relu'),
        Dropout(0.5),
        Dense(128, activation = 'relu'),
        Dropout(0.5),
        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.00001)
Conv2D_128V1_aug.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_128V1_aug.build(input_shape=(None, 128, 128, 1))

Conv2D_128V1_aug_history = Conv2D_128V1_aug.fit(
    train_128V2.batch(10), 
    epochs=200,
    validation_data=val_data_128.batch(10)
    )

Epoch 1/200
1433/1433 [==============================] - 9s 6ms/step - loss: 2.7083 - accuracy: 0.0704 - val_loss: 2.7057 - val_accuracy: 0.0707
Epoch 2/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.7064 - accuracy: 0.0711 - val_loss: 2.7036 - val_accuracy: 0.0673
Epoch 3/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.7045 - accuracy: 0.0753 - val_loss: 2.6963 - val_accuracy: 0.0803
Epoch 4/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.6884 - accuracy: 0.0966 - val_loss: 2.6332 - val_accuracy: 0.1870
Epoch 5/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.6021 - accuracy: 0.1331 - val_loss: 2.4680 - val_accuracy: 0.1717
Epoch 6/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.5252 - accuracy: 0.1493 - val_loss: 2.3945 - val_accuracy: 0.2103
Epoch 7/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.4813 - accuracy: 0.1636 - val_loss: 2.3479 - val_accuracy: 0.2353
Epoch 8/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.4418 - accuracy: 0.1733 - val_loss: 2.3236 - val_accuracy: 0.2323
Epoch 9/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.4148 - accuracy: 0.1785 - val_loss: 2.2692 - val_accuracy: 0.2537
Epoch 10/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.3671 - accuracy: 0.1897 - val_loss: 2.2417 - val_accuracy: 0.2517
Epoch 11/200
1433/1433 [==============================] - 8s 5ms/step - loss: 2.3339 - accuracy: 0.2075 - val_loss: 2.2614 - val_accuracy: 0.2370
Epoch 12/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.2851 - accuracy: 0.2259 - val_loss: 2.1566 - val_accuracy: 0.2847
Epoch 13/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.2353 - accuracy: 0.2435 - val_loss: 2.1494 - val_accuracy: 0.2757
Epoch 14/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.2002 - accuracy: 0.2603 - val_loss: 2.0544 - val_accuracy: 0.3160
Epoch 15/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.1612 - accuracy: 0.2716 - val_loss: 2.0464 - val_accuracy: 0.3083
Epoch 16/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.1279 - accuracy: 0.2779 - val_loss: 2.0433 - val_accuracy: 0.3043
Epoch 17/200
1433/1433 [==============================] - 8s 5ms/step - loss: 2.0847 - accuracy: 0.2994 - val_loss: 1.9801 - val_accuracy: 0.3347
Epoch 18/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.0654 - accuracy: 0.2999 - val_loss: 1.9040 - val_accuracy: 0.3560
Epoch 19/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.0388 - accuracy: 0.3127 - val_loss: 1.8899 - val_accuracy: 0.3727
Epoch 20/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.0182 - accuracy: 0.3236 - val_loss: 1.9367 - val_accuracy: 0.3490
Epoch 21/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.9784 - accuracy: 0.3308 - val_loss: 1.8505 - val_accuracy: 0.3817
Epoch 22/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.9626 - accuracy: 0.3384 - val_loss: 1.8232 - val_accuracy: 0.3873
Epoch 23/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.9440 - accuracy: 0.3442 - val_loss: 1.7900 - val_accuracy: 0.4000
Epoch 24/200
1433/1433 [==============================] - 8s 5ms/step - loss: 1.9128 - accuracy: 0.3485 - val_loss: 1.7681 - val_accuracy: 0.4013
Epoch 25/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.9062 - accuracy: 0.3548 - val_loss: 1.7767 - val_accuracy: 0.4053
Epoch 26/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.8703 - accuracy: 0.3675 - val_loss: 1.6890 - val_accuracy: 0.4330
Epoch 27/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.8537 - accuracy: 0.3729 - val_loss: 1.7332 - val_accuracy: 0.4173
Epoch 28/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.8305 - accuracy: 0.3814 - val_loss: 1.6356 - val_accuracy: 0.4537
Epoch 29/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.8238 - accuracy: 0.3843 - val_loss: 1.6242 - val_accuracy: 0.4520
Epoch 30/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.7824 - accuracy: 0.3964 - val_loss: 1.6488 - val_accuracy: 0.4443
Epoch 31/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.7674 - accuracy: 0.4019 - val_loss: 1.5678 - val_accuracy: 0.4697
Epoch 32/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.7461 - accuracy: 0.4114 - val_loss: 1.6176 - val_accuracy: 0.4453
Epoch 33/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.7255 - accuracy: 0.4178 - val_loss: 1.5757 - val_accuracy: 0.4687
Epoch 34/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.7013 - accuracy: 0.4289 - val_loss: 1.5542 - val_accuracy: 0.4750
Epoch 35/200
1433/1433 [==============================] - 9s 6ms/step - loss: 1.6963 - accuracy: 0.4292 - val_loss: 1.5189 - val_accuracy: 0.4853
Epoch 36/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.6627 - accuracy: 0.4431 - val_loss: 1.4664 - val_accuracy: 0.5050
Epoch 37/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.6329 - accuracy: 0.4484 - val_loss: 1.4543 - val_accuracy: 0.5073
Epoch 38/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.6140 - accuracy: 0.4579 - val_loss: 1.3771 - val_accuracy: 0.5283
Epoch 39/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.5967 - accuracy: 0.4564 - val_loss: 1.3886 - val_accuracy: 0.5250
Epoch 40/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.5654 - accuracy: 0.4676 - val_loss: 1.3841 - val_accuracy: 0.5267
Epoch 41/200
1433/1433 [==============================] - 8s 5ms/step - loss: 1.5571 - accuracy: 0.4755 - val_loss: 1.3466 - val_accuracy: 0.5413
Epoch 42/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.5328 - accuracy: 0.4850 - val_loss: 1.3533 - val_accuracy: 0.5383
Epoch 43/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.5085 - accuracy: 0.4928 - val_loss: 1.3497 - val_accuracy: 0.5397
Epoch 44/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.4792 - accuracy: 0.5074 - val_loss: 1.3909 - val_accuracy: 0.5300
Epoch 45/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.4615 - accuracy: 0.5093 - val_loss: 1.2842 - val_accuracy: 0.5647
Epoch 46/200
1433/1433 [==============================] - 8s 5ms/step - loss: 1.4497 - accuracy: 0.5139 - val_loss: 1.2130 - val_accuracy: 0.5873
Epoch 47/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.4370 - accuracy: 0.5239 - val_loss: 1.2353 - val_accuracy: 0.5863
Epoch 48/200
1433/1433 [==============================] - 8s 5ms/step - loss: 1.4171 - accuracy: 0.5291 - val_loss: 1.3293 - val_accuracy: 0.5547
Epoch 49/200
1433/1433 [==============================] - 8s 5ms/step - loss: 1.3944 - accuracy: 0.5384 - val_loss: 1.2083 - val_accuracy: 0.5947
Epoch 50/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.3810 - accuracy: 0.5394 - val_loss: 1.2461 - val_accuracy: 0.5877
Epoch 51/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.3520 - accuracy: 0.5520 - val_loss: 1.1539 - val_accuracy: 0.6163
Epoch 52/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.3489 - accuracy: 0.5503 - val_loss: 1.1823 - val_accuracy: 0.6043
Epoch 53/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.3240 - accuracy: 0.5643 - val_loss: 1.2518 - val_accuracy: 0.5893
Epoch 54/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.3226 - accuracy: 0.5610 - val_loss: 1.1506 - val_accuracy: 0.6140
Epoch 55/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.2933 - accuracy: 0.5700 - val_loss: 1.1570 - val_accuracy: 0.6170
Epoch 56/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.2781 - accuracy: 0.5758 - val_loss: 1.1942 - val_accuracy: 0.6067
Epoch 57/200
1433/1433 [==============================] - 8s 5ms/step - loss: 1.2659 - accuracy: 0.5876 - val_loss: 1.0850 - val_accuracy: 0.6440
Epoch 58/200
1433/1433 [==============================] - 8s 5ms/step - loss: 1.2524 - accuracy: 0.5881 - val_loss: 1.1354 - val_accuracy: 0.6273
Epoch 59/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.2345 - accuracy: 0.5964 - val_loss: 1.1440 - val_accuracy: 0.6250
Epoch 60/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.2163 - accuracy: 0.5987 - val_loss: 1.0813 - val_accuracy: 0.6477
Epoch 61/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.2047 - accuracy: 0.6052 - val_loss: 1.0336 - val_accuracy: 0.6603
Epoch 62/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.1866 - accuracy: 0.6067 - val_loss: 1.1015 - val_accuracy: 0.6430
Epoch 63/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.1722 - accuracy: 0.6127 - val_loss: 0.9779 - val_accuracy: 0.6843
Epoch 64/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.1682 - accuracy: 0.6119 - val_loss: 1.0568 - val_accuracy: 0.6587
Epoch 65/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.1624 - accuracy: 0.6219 - val_loss: 1.0715 - val_accuracy: 0.6577
Epoch 66/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.1342 - accuracy: 0.6249 - val_loss: 1.0325 - val_accuracy: 0.6667
Epoch 67/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.1261 - accuracy: 0.6293 - val_loss: 0.9914 - val_accuracy: 0.6810
Epoch 68/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.1132 - accuracy: 0.6353 - val_loss: 0.9617 - val_accuracy: 0.6897
Epoch 69/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.1070 - accuracy: 0.6369 - val_loss: 0.9629 - val_accuracy: 0.6883
Epoch 70/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0849 - accuracy: 0.6485 - val_loss: 0.9993 - val_accuracy: 0.6780
Epoch 71/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0761 - accuracy: 0.6464 - val_loss: 1.0290 - val_accuracy: 0.6627
Epoch 72/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0768 - accuracy: 0.6479 - val_loss: 0.9705 - val_accuracy: 0.6877
Epoch 73/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0511 - accuracy: 0.6586 - val_loss: 0.9278 - val_accuracy: 0.7030
Epoch 74/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0483 - accuracy: 0.6561 - val_loss: 0.9949 - val_accuracy: 0.6807
Epoch 75/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0339 - accuracy: 0.6616 - val_loss: 1.0429 - val_accuracy: 0.6683
Epoch 76/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0227 - accuracy: 0.6647 - val_loss: 0.9409 - val_accuracy: 0.7020
Epoch 77/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0108 - accuracy: 0.6739 - val_loss: 0.9326 - val_accuracy: 0.7030
Epoch 78/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0019 - accuracy: 0.6735 - val_loss: 0.9263 - val_accuracy: 0.7000
Epoch 79/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.9888 - accuracy: 0.6751 - val_loss: 0.9088 - val_accuracy: 0.7067
Epoch 80/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.9908 - accuracy: 0.6787 - val_loss: 0.8516 - val_accuracy: 0.7250
Epoch 81/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.9643 - accuracy: 0.6887 - val_loss: 0.8688 - val_accuracy: 0.7200
Epoch 82/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.9537 - accuracy: 0.6877 - val_loss: 0.8913 - val_accuracy: 0.7123
Epoch 83/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.9482 - accuracy: 0.6921 - val_loss: 0.8787 - val_accuracy: 0.7210
Epoch 84/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.9423 - accuracy: 0.6923 - val_loss: 0.8098 - val_accuracy: 0.7440
Epoch 85/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.9319 - accuracy: 0.6995 - val_loss: 0.9031 - val_accuracy: 0.7040
Epoch 86/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.9197 - accuracy: 0.7027 - val_loss: 0.8779 - val_accuracy: 0.7123
Epoch 87/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.9132 - accuracy: 0.7010 - val_loss: 0.8735 - val_accuracy: 0.7167
Epoch 88/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.8950 - accuracy: 0.7075 - val_loss: 0.7695 - val_accuracy: 0.7547
Epoch 89/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.9005 - accuracy: 0.7032 - val_loss: 0.9102 - val_accuracy: 0.7053
Epoch 90/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.8822 - accuracy: 0.7178 - val_loss: 0.8054 - val_accuracy: 0.7407
Epoch 91/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.8849 - accuracy: 0.7159 - val_loss: 0.8132 - val_accuracy: 0.7343
Epoch 92/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.8721 - accuracy: 0.7187 - val_loss: 0.7697 - val_accuracy: 0.7520
Epoch 93/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.8529 - accuracy: 0.7234 - val_loss: 0.7974 - val_accuracy: 0.7410
Epoch 94/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.8521 - accuracy: 0.7258 - val_loss: 0.9309 - val_accuracy: 0.7010
Epoch 95/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.8484 - accuracy: 0.7292 - val_loss: 0.7936 - val_accuracy: 0.7440
Epoch 96/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.8405 - accuracy: 0.7247 - val_loss: 0.8240 - val_accuracy: 0.7317
Epoch 97/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.8291 - accuracy: 0.7304 - val_loss: 0.8767 - val_accuracy: 0.7160
Epoch 98/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.8145 - accuracy: 0.7359 - val_loss: 0.7538 - val_accuracy: 0.7567
Epoch 99/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.8137 - accuracy: 0.7326 - val_loss: 0.7811 - val_accuracy: 0.7440
Epoch 100/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.8083 - accuracy: 0.7390 - val_loss: 0.8084 - val_accuracy: 0.7323
Epoch 101/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.7992 - accuracy: 0.7402 - val_loss: 0.7887 - val_accuracy: 0.7427
Epoch 102/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.7964 - accuracy: 0.7421 - val_loss: 0.7810 - val_accuracy: 0.7423
Epoch 103/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.7845 - accuracy: 0.7433 - val_loss: 0.7562 - val_accuracy: 0.7470
Epoch 104/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.7718 - accuracy: 0.7518 - val_loss: 0.7300 - val_accuracy: 0.7597
Epoch 105/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.7731 - accuracy: 0.7492 - val_loss: 0.7189 - val_accuracy: 0.7650
Epoch 106/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.7684 - accuracy: 0.7507 - val_loss: 0.7292 - val_accuracy: 0.7593
Epoch 107/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.7496 - accuracy: 0.7590 - val_loss: 0.7103 - val_accuracy: 0.7663
Epoch 108/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.7432 - accuracy: 0.7622 - val_loss: 0.6973 - val_accuracy: 0.7757
Epoch 109/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.7470 - accuracy: 0.7588 - val_loss: 0.7250 - val_accuracy: 0.7637
Epoch 110/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.7324 - accuracy: 0.7617 - val_loss: 0.7079 - val_accuracy: 0.7703
Epoch 111/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.7241 - accuracy: 0.7638 - val_loss: 0.7422 - val_accuracy: 0.7577
Epoch 112/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.7180 - accuracy: 0.7698 - val_loss: 0.6901 - val_accuracy: 0.7783
Epoch 113/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.7140 - accuracy: 0.7681 - val_loss: 0.7304 - val_accuracy: 0.7637
Epoch 114/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.7110 - accuracy: 0.7689 - val_loss: 0.6788 - val_accuracy: 0.7757
Epoch 115/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.7070 - accuracy: 0.7737 - val_loss: 0.6676 - val_accuracy: 0.7820
Epoch 116/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6877 - accuracy: 0.7797 - val_loss: 0.6969 - val_accuracy: 0.7717
Epoch 117/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.6899 - accuracy: 0.7800 - val_loss: 0.6637 - val_accuracy: 0.7830
Epoch 118/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.6813 - accuracy: 0.7786 - val_loss: 0.6794 - val_accuracy: 0.7767
Epoch 119/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.6757 - accuracy: 0.7835 - val_loss: 0.6570 - val_accuracy: 0.7840
Epoch 120/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6778 - accuracy: 0.7804 - val_loss: 0.8034 - val_accuracy: 0.7350
Epoch 121/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6688 - accuracy: 0.7844 - val_loss: 0.6468 - val_accuracy: 0.7893
Epoch 122/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.6496 - accuracy: 0.7904 - val_loss: 0.7303 - val_accuracy: 0.7597
Epoch 123/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6519 - accuracy: 0.7881 - val_loss: 0.7486 - val_accuracy: 0.7507
Epoch 124/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6402 - accuracy: 0.7961 - val_loss: 0.6793 - val_accuracy: 0.7747
Epoch 125/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.6438 - accuracy: 0.7930 - val_loss: 0.6201 - val_accuracy: 0.7930
Epoch 126/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.6439 - accuracy: 0.7908 - val_loss: 0.6538 - val_accuracy: 0.7797
Epoch 127/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.6308 - accuracy: 0.7935 - val_loss: 0.6189 - val_accuracy: 0.7907
Epoch 128/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6244 - accuracy: 0.7964 - val_loss: 0.6699 - val_accuracy: 0.7780
Epoch 129/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6123 - accuracy: 0.8017 - val_loss: 0.7074 - val_accuracy: 0.7680
Epoch 130/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6190 - accuracy: 0.8016 - val_loss: 0.6613 - val_accuracy: 0.7760
Epoch 131/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6006 - accuracy: 0.8025 - val_loss: 0.6471 - val_accuracy: 0.7843
Epoch 132/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6078 - accuracy: 0.8021 - val_loss: 0.6388 - val_accuracy: 0.7917
Epoch 133/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5948 - accuracy: 0.8073 - val_loss: 0.5923 - val_accuracy: 0.8060
Epoch 134/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.6044 - accuracy: 0.8026 - val_loss: 0.6292 - val_accuracy: 0.7920
Epoch 135/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5831 - accuracy: 0.8118 - val_loss: 0.5781 - val_accuracy: 0.8123
Epoch 136/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5780 - accuracy: 0.8136 - val_loss: 0.5905 - val_accuracy: 0.8080
Epoch 137/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5752 - accuracy: 0.8165 - val_loss: 0.5937 - val_accuracy: 0.8067
Epoch 138/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5600 - accuracy: 0.8181 - val_loss: 0.6108 - val_accuracy: 0.7967
Epoch 139/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5734 - accuracy: 0.8175 - val_loss: 0.6121 - val_accuracy: 0.8037
Epoch 140/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5641 - accuracy: 0.8182 - val_loss: 0.5873 - val_accuracy: 0.8077
Epoch 141/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5640 - accuracy: 0.8177 - val_loss: 0.5737 - val_accuracy: 0.8180
Epoch 142/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.5507 - accuracy: 0.8246 - val_loss: 0.6142 - val_accuracy: 0.8003
Epoch 143/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5475 - accuracy: 0.8242 - val_loss: 0.5136 - val_accuracy: 0.8353
Epoch 144/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5508 - accuracy: 0.8268 - val_loss: 0.5789 - val_accuracy: 0.8097
Epoch 145/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5379 - accuracy: 0.8246 - val_loss: 0.5425 - val_accuracy: 0.8217
Epoch 146/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5310 - accuracy: 0.8249 - val_loss: 0.6025 - val_accuracy: 0.8030
Epoch 147/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5236 - accuracy: 0.8320 - val_loss: 0.6539 - val_accuracy: 0.7873
Epoch 148/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5235 - accuracy: 0.8288 - val_loss: 0.5298 - val_accuracy: 0.8283
Epoch 149/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.5246 - accuracy: 0.8316 - val_loss: 0.5427 - val_accuracy: 0.8220
Epoch 150/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.5172 - accuracy: 0.8326 - val_loss: 0.5115 - val_accuracy: 0.8350
Epoch 151/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5100 - accuracy: 0.8355 - val_loss: 0.5785 - val_accuracy: 0.8100
Epoch 152/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.5113 - accuracy: 0.8377 - val_loss: 0.5837 - val_accuracy: 0.8107
Epoch 153/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5060 - accuracy: 0.8395 - val_loss: 0.5253 - val_accuracy: 0.8270
Epoch 154/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4957 - accuracy: 0.8411 - val_loss: 0.5742 - val_accuracy: 0.8113
Epoch 155/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.4841 - accuracy: 0.8432 - val_loss: 0.6323 - val_accuracy: 0.7957
Epoch 156/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.4898 - accuracy: 0.8406 - val_loss: 0.5383 - val_accuracy: 0.8213
Epoch 157/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4858 - accuracy: 0.8390 - val_loss: 0.5329 - val_accuracy: 0.8220
Epoch 158/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.4723 - accuracy: 0.8459 - val_loss: 0.5036 - val_accuracy: 0.8337
Epoch 159/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.4810 - accuracy: 0.8454 - val_loss: 0.5341 - val_accuracy: 0.8310
Epoch 160/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4773 - accuracy: 0.8429 - val_loss: 0.6018 - val_accuracy: 0.8003
Epoch 161/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.4741 - accuracy: 0.8491 - val_loss: 0.5598 - val_accuracy: 0.8157
Epoch 162/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.4543 - accuracy: 0.8526 - val_loss: 0.5136 - val_accuracy: 0.8373
Epoch 163/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4636 - accuracy: 0.8493 - val_loss: 0.5315 - val_accuracy: 0.8293
Epoch 164/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.4622 - accuracy: 0.8528 - val_loss: 0.4868 - val_accuracy: 0.8417
Epoch 165/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.4540 - accuracy: 0.8510 - val_loss: 0.5140 - val_accuracy: 0.8327
Epoch 166/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4427 - accuracy: 0.8552 - val_loss: 0.5503 - val_accuracy: 0.8230
Epoch 167/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4415 - accuracy: 0.8584 - val_loss: 0.5382 - val_accuracy: 0.8260
Epoch 168/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.4445 - accuracy: 0.8562 - val_loss: 0.5700 - val_accuracy: 0.8160
Epoch 169/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.4290 - accuracy: 0.8637 - val_loss: 0.4745 - val_accuracy: 0.8473
Epoch 170/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4379 - accuracy: 0.8591 - val_loss: 0.4448 - val_accuracy: 0.8553
Epoch 171/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4323 - accuracy: 0.8586 - val_loss: 0.4914 - val_accuracy: 0.8433
Epoch 172/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4189 - accuracy: 0.8650 - val_loss: 0.6788 - val_accuracy: 0.7893
Epoch 173/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4124 - accuracy: 0.8638 - val_loss: 0.4933 - val_accuracy: 0.8390
Epoch 174/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4242 - accuracy: 0.8638 - val_loss: 0.4559 - val_accuracy: 0.8547
Epoch 175/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4121 - accuracy: 0.8647 - val_loss: 0.4250 - val_accuracy: 0.8620
Epoch 176/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4120 - accuracy: 0.8635 - val_loss: 0.4591 - val_accuracy: 0.8520
Epoch 177/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4132 - accuracy: 0.8646 - val_loss: 0.4852 - val_accuracy: 0.8433
Epoch 178/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4040 - accuracy: 0.8677 - val_loss: 0.5116 - val_accuracy: 0.8370
Epoch 179/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4034 - accuracy: 0.8697 - val_loss: 0.4913 - val_accuracy: 0.8433
Epoch 180/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3923 - accuracy: 0.8725 - val_loss: 0.5055 - val_accuracy: 0.8390
Epoch 181/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3962 - accuracy: 0.8734 - val_loss: 0.5012 - val_accuracy: 0.8447
Epoch 182/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3998 - accuracy: 0.8716 - val_loss: 0.4884 - val_accuracy: 0.8423
Epoch 183/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3829 - accuracy: 0.8773 - val_loss: 0.4674 - val_accuracy: 0.8537
Epoch 184/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3861 - accuracy: 0.8753 - val_loss: 0.4596 - val_accuracy: 0.8560
Epoch 185/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3773 - accuracy: 0.8792 - val_loss: 0.4831 - val_accuracy: 0.8510
Epoch 186/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3846 - accuracy: 0.8773 - val_loss: 0.4589 - val_accuracy: 0.8567
Epoch 187/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3742 - accuracy: 0.8776 - val_loss: 0.4951 - val_accuracy: 0.8400
Epoch 188/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3718 - accuracy: 0.8800 - val_loss: 0.4409 - val_accuracy: 0.8630
Epoch 189/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3692 - accuracy: 0.8820 - val_loss: 0.4960 - val_accuracy: 0.8423
Epoch 190/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3666 - accuracy: 0.8822 - val_loss: 0.4140 - val_accuracy: 0.8690
Epoch 191/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3550 - accuracy: 0.8868 - val_loss: 0.4556 - val_accuracy: 0.8590
Epoch 192/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3533 - accuracy: 0.8829 - val_loss: 0.4751 - val_accuracy: 0.8510
Epoch 193/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3615 - accuracy: 0.8817 - val_loss: 0.4227 - val_accuracy: 0.8680
Epoch 194/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3546 - accuracy: 0.8864 - val_loss: 0.5352 - val_accuracy: 0.8340
Epoch 195/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3517 - accuracy: 0.8849 - val_loss: 0.5157 - val_accuracy: 0.8393
Epoch 196/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3460 - accuracy: 0.8887 - val_loss: 0.4310 - val_accuracy: 0.8617
Epoch 197/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3356 - accuracy: 0.8916 - val_loss: 0.4590 - val_accuracy: 0.8543
Epoch 198/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.3426 - accuracy: 0.8887 - val_loss: 0.4213 - val_accuracy: 0.8663
Epoch 199/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3363 - accuracy: 0.8880 - val_loss: 0.4406 - val_accuracy: 0.8600
Epoch 200/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.3356 - accuracy: 0.8910 - val_loss: 0.5450 - val_accuracy: 0.8313

Conv2D_128V1_aug.summary()

Model: "Conv2D_128V1_Augmentation"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalised_data (Sequential  (None, None, None, 1)    0         
 )                                                               
                                                                 
 conv2d (Conv2D)             (None, 32, 32, 64)        1664      
                                                                 
 max_pooling2d (MaxPooling2D  (None, 16, 16, 64)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 8, 8, 128)        0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 4, 4, 128)        0         
 2D)                                                             
                                                                 
 conv2d_3 (Conv2D)           (None, 4, 4, 256)         295168    
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 2, 2, 256)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 1024)              0         
                                                                 
 dropout (Dropout)           (None, 1024)              0         
                                                                 
 dense (Dense)               (None, 1024)              1049600   
                                                                 
 dropout_1 (Dropout)         (None, 1024)              0         
                                                                 
 dense_1 (Dense)             (None, 128)               131200    
                                                                 
 dropout_2 (Dropout)         (None, 128)               0         
                                                                 
 dense_2 (Dense)             (None, 15)                1935      
                                                                 
=================================================================
Total params: 1,701,007
Trainable params: 1,701,007
Non-trainable params: 0
_________________________________________________________________

plot_learning_curve(Conv2D_128V1_aug_history.history)

Observations

• This model still has room for improvement as the training accuracy only reaches 0.89 hence maybe an increase in epochs or a decrease in learning rate will increase the accuracy.
• Validation loss and accuracy is quite close towards the training loss and accuracy which shows that the model is quite reliable.
• Compared to the unaugmented data, the accuracy curve and loss curve of the validation data does not diverge away from the training curve that much, which shows that augmenting the data does improve the prediction of the model.

Conv2D_128V1_aug.evaluate(test_data_128.batch(10))

300/300 [==============================] - 3s 7ms/step - loss: 0.5392 - accuracy: 0.8360

[0.5391502976417542, 0.8360000252723694]

Observations

• Test accuracy of 0.80+ is relatively quite high.

Transfer Learning - ResNet50

ResNet50 is a CNN that is 50 layers deep, a kind that stacks residual blocks on top of each other to form a network.

While deep CNNs gives us the option of adding more layers to the CNNs to solve more complicated tasks in computer vision, it comes with its own set of issues. It has been observed that training the neural networks becomes more difficult with the increase in the number of added layers, and in some cases, the accuracy dwindles as well. It is here that the use of ResNet assumes importance. Deeper neural networks are more difficult to train. With Resnet, it becomes possible to surpass the difficulties of training very deep neural networks.

For computer vision tasks, experts often add more layers to enhance the model's capability to solve complex problem effectively. The idea is that each layer can be specialized to learn specific features, leading to improved accuracy. However, as the number of layers increases, a phenomenon called "degradation" may occur. This means that, despite having a deeper network, the performance may reach a point where further increases in depth lead to a decline in accuracy on both training and testing data. It may result from various factors such as the initialization of the network parameters, the choice of optimization functions or a serious issue also known as vanishing or exploding gradients.

Vanishing gradient - Gradient of the loss function become extremely small during back propagation, hindering effective weight updates and learning.
Exploding gradient - Excessively large gradients that cause unstable learning.
Read more at: https://viso.ai/deep-learning/resnet-residual-neural-network/

Residual blocks are used in ResNet to improve the accuracy which involves the concept of 'skip connection'.

Skip connections work in two ways. They alleviate the issue of vanishing gradient by setting up an alternate shortcute for the gradient to pass through which enable the model to learn an identity function so that the higher layers of the model do not perform worse than the lower layers. ResNet will thus improve the efficiency of deep neural networks with more neural layers while minimizing percentage of errors.

# Try for 128 x 128 images
tf.keras.backend.clear_session()

ResNet50_128 = Sequential(name="ResNet50_Augmentation",
    layers = [
        ResNet50(
            include_top=False, 
            weights=None, 
            input_shape=(128, 128, 1),
            classifier_activation="softmax"),
        
        GlobalAveragePooling2D(),
        Dropout(0.5),
        Dense(512, activation = 'relu'),
        Dropout(0.5),
        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.0001)
ResNet50_128.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

ResNet50_128.build(input_shape=(None, 128, 128, 1))

ResNet50_128_history = ResNet50_128.fit(
    train_128V2.batch(10), 
    epochs=100,
    validation_data=val_data_128.batch(10)
    )

Epoch 1/100
1433/1433 [==============================] - 43s 28ms/step - loss: 2.7421 - accuracy: 0.1270 - val_loss: 2.2832 - val_accuracy: 0.2473
Epoch 2/100
1433/1433 [==============================] - 39s 27ms/step - loss: 2.1871 - accuracy: 0.2795 - val_loss: 2.0296 - val_accuracy: 0.3910
Epoch 3/100
1433/1433 [==============================] - 39s 27ms/step - loss: 1.8260 - accuracy: 0.4062 - val_loss: 2.3629 - val_accuracy: 0.3747
Epoch 4/100
1433/1433 [==============================] - 38s 27ms/step - loss: 1.5579 - accuracy: 0.4931 - val_loss: 1.3222 - val_accuracy: 0.5787
Epoch 5/100
1433/1433 [==============================] - 38s 27ms/step - loss: 1.3158 - accuracy: 0.5784 - val_loss: 1.2047 - val_accuracy: 0.6577
Epoch 6/100
1433/1433 [==============================] - 38s 27ms/step - loss: 1.0958 - accuracy: 0.6584 - val_loss: 1.1866 - val_accuracy: 0.6640
Epoch 7/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.8613 - accuracy: 0.7366 - val_loss: 3.2008 - val_accuracy: 0.4083
Epoch 8/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.6779 - accuracy: 0.7909 - val_loss: 1.9312 - val_accuracy: 0.5817
Epoch 9/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.5236 - accuracy: 0.8375 - val_loss: 0.9294 - val_accuracy: 0.7637
Epoch 10/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.4468 - accuracy: 0.8644 - val_loss: 0.9722 - val_accuracy: 0.7850
Epoch 11/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.3661 - accuracy: 0.8890 - val_loss: 0.7258 - val_accuracy: 0.8227
Epoch 12/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.2990 - accuracy: 0.9109 - val_loss: 1.1325 - val_accuracy: 0.7720
Epoch 13/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.2557 - accuracy: 0.9232 - val_loss: 0.4938 - val_accuracy: 0.8793
Epoch 14/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.2218 - accuracy: 0.9328 - val_loss: 1.0242 - val_accuracy: 0.8263
Epoch 15/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.2042 - accuracy: 0.9383 - val_loss: 0.5062 - val_accuracy: 0.8820
Epoch 16/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.1968 - accuracy: 0.9419 - val_loss: 10.2790 - val_accuracy: 0.3117
Epoch 17/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.1685 - accuracy: 0.9497 - val_loss: 0.5121 - val_accuracy: 0.8907
Epoch 18/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.1583 - accuracy: 0.9520 - val_loss: 0.9357 - val_accuracy: 0.8153
Epoch 19/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.1463 - accuracy: 0.9561 - val_loss: 0.5661 - val_accuracy: 0.8790
Epoch 20/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.1263 - accuracy: 0.9610 - val_loss: 2.6327 - val_accuracy: 0.6150
Epoch 21/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.1373 - accuracy: 0.9600 - val_loss: 2.0516 - val_accuracy: 0.7337
Epoch 22/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.1231 - accuracy: 0.9635 - val_loss: 0.6639 - val_accuracy: 0.8640
Epoch 23/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.1175 - accuracy: 0.9651 - val_loss: 1.1632 - val_accuracy: 0.8073
Epoch 24/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.1009 - accuracy: 0.9705 - val_loss: 9.8689 - val_accuracy: 0.3397
Epoch 25/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.1069 - accuracy: 0.9696 - val_loss: 2.1818 - val_accuracy: 0.6717
Epoch 26/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0921 - accuracy: 0.9738 - val_loss: 1.7106 - val_accuracy: 0.7760
Epoch 27/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0899 - accuracy: 0.9754 - val_loss: 1.3592 - val_accuracy: 0.7433
Epoch 28/100
1433/1433 [==============================] - 40s 28ms/step - loss: 0.0906 - accuracy: 0.9718 - val_loss: 0.7329 - val_accuracy: 0.8740
Epoch 29/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0865 - accuracy: 0.9753 - val_loss: 14.5525 - val_accuracy: 0.2503
Epoch 30/100
1433/1433 [==============================] - 39s 28ms/step - loss: 0.0814 - accuracy: 0.9764 - val_loss: 0.5942 - val_accuracy: 0.8873
Epoch 31/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0678 - accuracy: 0.9803 - val_loss: 0.5883 - val_accuracy: 0.8830
Epoch 32/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0791 - accuracy: 0.9784 - val_loss: 1.2076 - val_accuracy: 0.8617
Epoch 33/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0710 - accuracy: 0.9797 - val_loss: 0.6542 - val_accuracy: 0.8820
Epoch 34/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0691 - accuracy: 0.9813 - val_loss: 1.0447 - val_accuracy: 0.8033
Epoch 35/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0629 - accuracy: 0.9819 - val_loss: 2.3741 - val_accuracy: 0.7457
Epoch 36/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0678 - accuracy: 0.9809 - val_loss: 0.4118 - val_accuracy: 0.9213
Epoch 37/100
1433/1433 [==============================] - 40s 28ms/step - loss: 0.0740 - accuracy: 0.9802 - val_loss: 1.1953 - val_accuracy: 0.8357
Epoch 38/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0620 - accuracy: 0.9842 - val_loss: 0.7873 - val_accuracy: 0.8783
Epoch 39/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0645 - accuracy: 0.9813 - val_loss: 3.8473 - val_accuracy: 0.6733
Epoch 40/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0537 - accuracy: 0.9847 - val_loss: 5.0184 - val_accuracy: 0.5230
Epoch 41/100
1433/1433 [==============================] - 40s 28ms/step - loss: 0.0625 - accuracy: 0.9805 - val_loss: 2.2644 - val_accuracy: 0.7733
Epoch 42/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0576 - accuracy: 0.9837 - val_loss: 3.9936 - val_accuracy: 0.5810
Epoch 43/100
1433/1433 [==============================] - 37s 26ms/step - loss: 0.0525 - accuracy: 0.9858 - val_loss: 0.3981 - val_accuracy: 0.9353
Epoch 44/100
1433/1433 [==============================] - 37s 26ms/step - loss: 0.0557 - accuracy: 0.9861 - val_loss: 3.0441 - val_accuracy: 0.7077
Epoch 45/100
1433/1433 [==============================] - 37s 26ms/step - loss: 0.0482 - accuracy: 0.9869 - val_loss: 0.3628 - val_accuracy: 0.9340
Epoch 46/100
1433/1433 [==============================] - 37s 26ms/step - loss: 0.0571 - accuracy: 0.9851 - val_loss: 0.4014 - val_accuracy: 0.9333
Epoch 47/100
1433/1433 [==============================] - 37s 26ms/step - loss: 0.0451 - accuracy: 0.9877 - val_loss: 1.8447 - val_accuracy: 0.7710
Epoch 48/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0474 - accuracy: 0.9876 - val_loss: 1.8113 - val_accuracy: 0.7400
Epoch 49/100
1433/1433 [==============================] - 40s 28ms/step - loss: 0.0439 - accuracy: 0.9871 - val_loss: 0.3017 - val_accuracy: 0.9370
Epoch 50/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0527 - accuracy: 0.9860 - val_loss: 5.2311 - val_accuracy: 0.6163
Epoch 51/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0517 - accuracy: 0.9869 - val_loss: 0.4404 - val_accuracy: 0.9297
Epoch 52/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0417 - accuracy: 0.9884 - val_loss: 4.8343 - val_accuracy: 0.5103
Epoch 53/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0506 - accuracy: 0.9869 - val_loss: 1.4589 - val_accuracy: 0.8117
Epoch 54/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0496 - accuracy: 0.9870 - val_loss: 0.7719 - val_accuracy: 0.8640
Epoch 55/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0378 - accuracy: 0.9894 - val_loss: 0.7448 - val_accuracy: 0.8800
Epoch 56/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0415 - accuracy: 0.9883 - val_loss: 0.5286 - val_accuracy: 0.9110
Epoch 57/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0399 - accuracy: 0.9892 - val_loss: 0.3801 - val_accuracy: 0.9230
Epoch 58/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0409 - accuracy: 0.9890 - val_loss: 1.6336 - val_accuracy: 0.8027
Epoch 59/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0417 - accuracy: 0.9887 - val_loss: 1.3666 - val_accuracy: 0.8023
Epoch 60/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0343 - accuracy: 0.9913 - val_loss: 4.0972 - val_accuracy: 0.5683
Epoch 61/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0410 - accuracy: 0.9888 - val_loss: 1.0047 - val_accuracy: 0.8483
Epoch 62/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0260 - accuracy: 0.9934 - val_loss: 0.4476 - val_accuracy: 0.9280
Epoch 63/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0404 - accuracy: 0.9889 - val_loss: 0.7441 - val_accuracy: 0.8860
Epoch 64/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0320 - accuracy: 0.9918 - val_loss: 0.4775 - val_accuracy: 0.9220
Epoch 65/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0375 - accuracy: 0.9902 - val_loss: 0.4740 - val_accuracy: 0.9347
Epoch 66/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0384 - accuracy: 0.9900 - val_loss: 1.0379 - val_accuracy: 0.8913
Epoch 67/100
1433/1433 [==============================] - 40s 28ms/step - loss: 0.0433 - accuracy: 0.9897 - val_loss: 0.8277 - val_accuracy: 0.8610
Epoch 68/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0262 - accuracy: 0.9936 - val_loss: 1.6325 - val_accuracy: 0.7800
Epoch 69/100
1433/1433 [==============================] - 40s 28ms/step - loss: 0.0326 - accuracy: 0.9911 - val_loss: 0.7333 - val_accuracy: 0.8890
Epoch 70/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0364 - accuracy: 0.9904 - val_loss: 0.4073 - val_accuracy: 0.9347
Epoch 71/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0421 - accuracy: 0.9899 - val_loss: 8.2934 - val_accuracy: 0.4543
Epoch 72/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0264 - accuracy: 0.9931 - val_loss: 1.2749 - val_accuracy: 0.8670
Epoch 73/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0141 - accuracy: 0.9957 - val_loss: 0.5470 - val_accuracy: 0.9193
Epoch 74/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0528 - accuracy: 0.9876 - val_loss: 3.2790 - val_accuracy: 0.7023
Epoch 75/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0270 - accuracy: 0.9918 - val_loss: 0.5196 - val_accuracy: 0.9173
Epoch 76/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0179 - accuracy: 0.9955 - val_loss: 2.0091 - val_accuracy: 0.8443
Epoch 77/100
1433/1433 [==============================] - 39s 28ms/step - loss: 0.0516 - accuracy: 0.9885 - val_loss: 0.7187 - val_accuracy: 0.8900
Epoch 78/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0127 - accuracy: 0.9962 - val_loss: 0.3767 - val_accuracy: 0.9377
Epoch 79/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0405 - accuracy: 0.9908 - val_loss: 1.5927 - val_accuracy: 0.8213
Epoch 80/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0320 - accuracy: 0.9918 - val_loss: 1.6802 - val_accuracy: 0.8070
Epoch 81/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0282 - accuracy: 0.9927 - val_loss: 0.3404 - val_accuracy: 0.9447
Epoch 82/100
1433/1433 [==============================] - 38s 26ms/step - loss: 0.0369 - accuracy: 0.9916 - val_loss: 0.4395 - val_accuracy: 0.9380
Epoch 83/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0299 - accuracy: 0.9922 - val_loss: 1.0202 - val_accuracy: 0.8647
Epoch 84/100
1433/1433 [==============================] - 40s 28ms/step - loss: 0.0259 - accuracy: 0.9941 - val_loss: 1.7208 - val_accuracy: 0.7827
Epoch 85/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0209 - accuracy: 0.9938 - val_loss: 0.4812 - val_accuracy: 0.9240
Epoch 86/100
1433/1433 [==============================] - 39s 28ms/step - loss: 0.0311 - accuracy: 0.9923 - val_loss: 0.5283 - val_accuracy: 0.9163
Epoch 87/100
1433/1433 [==============================] - 39s 28ms/step - loss: 0.0255 - accuracy: 0.9939 - val_loss: 0.4890 - val_accuracy: 0.9157
Epoch 88/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0334 - accuracy: 0.9923 - val_loss: 0.3024 - val_accuracy: 0.9490
Epoch 89/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0105 - accuracy: 0.9967 - val_loss: 1.5105 - val_accuracy: 0.8797
Epoch 90/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0443 - accuracy: 0.9889 - val_loss: 0.4322 - val_accuracy: 0.9297
Epoch 91/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0261 - accuracy: 0.9943 - val_loss: 0.3805 - val_accuracy: 0.9427
Epoch 92/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0148 - accuracy: 0.9957 - val_loss: 0.6444 - val_accuracy: 0.9110
Epoch 93/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0274 - accuracy: 0.9928 - val_loss: 1.6098 - val_accuracy: 0.7983
Epoch 94/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0243 - accuracy: 0.9945 - val_loss: 15.9933 - val_accuracy: 0.2790
Epoch 95/100
1433/1433 [==============================] - 40s 28ms/step - loss: 0.0275 - accuracy: 0.9918 - val_loss: 0.6209 - val_accuracy: 0.9150
Epoch 96/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0265 - accuracy: 0.9937 - val_loss: 1.3750 - val_accuracy: 0.8340
Epoch 97/100
1433/1433 [==============================] - 39s 27ms/step - loss: 0.0255 - accuracy: 0.9934 - val_loss: 0.4504 - val_accuracy: 0.9307
Epoch 98/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0164 - accuracy: 0.9952 - val_loss: 0.8724 - val_accuracy: 0.8933
Epoch 99/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0298 - accuracy: 0.9928 - val_loss: 1.7868 - val_accuracy: 0.8207
Epoch 100/100
1433/1433 [==============================] - 38s 27ms/step - loss: 0.0145 - accuracy: 0.9963 - val_loss: 0.3554 - val_accuracy: 0.9443

ResNet50_128.summary()

Model: "ResNet50_Augmentation"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 resnet50 (Functional)       (None, 4, 4, 2048)        23581440  
                                                                 
 global_average_pooling2d (G  (None, 2048)             0         
 lobalAveragePooling2D)                                          
                                                                 
 dropout (Dropout)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 512)               1049088   
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_1 (Dense)             (None, 15)                7695      
                                                                 
=================================================================
Total params: 24,638,223
Trainable params: 24,585,103
Non-trainable params: 53,120
_________________________________________________________________

plot_learning_curve(ResNet50_128_history.history)

Observations

• ResNet50 shows that the model is too accustomed with the training data where the training accuracy is close to 1.0 and the training loss is apporaching near 0.
• Constant fluctuations can be seen in both the validation curves.
• Requires deeper research if were to implement transfer learning

ResNet50_128.evaluate(test_data_128.batch(10))

300/300 [==============================] - 3s 9ms/step - loss: 0.4457 - accuracy: 0.9417

[0.4457317292690277, 0.9416666626930237]

Observations

• Test accuracy is still high but due to the curve showing fluctuations, we can't use ResNet50.

31 x 31 images

CNN Augmented Balance Version 1 (31 x 31)

# Try for 31 x 31 images
tf.keras.backend.clear_session()

Conv2D_31_aug = Sequential(name="Conv2D_31_Augmentation",
    layers = [
        normalised_data,
        Conv2D(32, (3, 3),input_shape=(31, 31 ,1), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),
        Conv2D(64, (3, 3), activation='relu', padding='same'),
        Conv2D(128, (3, 3), activation='relu', padding='same'),
        Conv2D(128, (3, 3), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.5),

        Dense(512, activation='relu'),
        Dropout(0.5),
        Dense(256, activation='relu'),
        Dropout(0.5),
        # Dense(128, activation='relu'),
        # Dropout(0.5),

        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.0001)
Conv2D_31_aug.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_31_aug.build(input_shape=(None, 31, 31, 1))

Conv2D_31_aug_history = Conv2D_31_aug.fit(
    train_31V2.batch(10), 
    epochs=200,
    validation_data=val_data_31.batch(10)
    )

Epoch 1/200
1433/1433 [==============================] - 37s 26ms/step - loss: 2.6103 - accuracy: 0.1169 - val_loss: 2.5959 - val_accuracy: 0.1560
Epoch 2/200
1433/1433 [==============================] - 37s 26ms/step - loss: 2.2159 - accuracy: 0.2665 - val_loss: 2.0890 - val_accuracy: 0.3087
Epoch 3/200
1433/1433 [==============================] - 37s 26ms/step - loss: 1.9160 - accuracy: 0.3652 - val_loss: 1.7975 - val_accuracy: 0.4023
Epoch 4/200
1433/1433 [==============================] - 37s 26ms/step - loss: 1.6669 - accuracy: 0.4566 - val_loss: 1.6381 - val_accuracy: 0.4637
Epoch 5/200
1433/1433 [==============================] - 35s 25ms/step - loss: 1.4602 - accuracy: 0.5303 - val_loss: 1.6507 - val_accuracy: 0.4780
Epoch 6/200
1433/1433 [==============================] - 35s 24ms/step - loss: 1.2915 - accuracy: 0.5824 - val_loss: 1.3950 - val_accuracy: 0.5593
Epoch 7/200
1433/1433 [==============================] - 35s 24ms/step - loss: 1.1555 - accuracy: 0.6329 - val_loss: 1.3091 - val_accuracy: 0.5893
Epoch 8/200
1433/1433 [==============================] - 34s 24ms/step - loss: 1.0240 - accuracy: 0.6713 - val_loss: 1.0064 - val_accuracy: 0.6767
Epoch 9/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.9218 - accuracy: 0.7036 - val_loss: 0.9894 - val_accuracy: 0.6807
Epoch 10/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.8307 - accuracy: 0.7347 - val_loss: 0.9974 - val_accuracy: 0.6787
Epoch 11/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.7628 - accuracy: 0.7534 - val_loss: 1.0042 - val_accuracy: 0.6803
Epoch 12/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.6834 - accuracy: 0.7807 - val_loss: 1.1573 - val_accuracy: 0.6597
Epoch 13/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.6264 - accuracy: 0.7962 - val_loss: 0.9360 - val_accuracy: 0.6970
Epoch 14/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.5835 - accuracy: 0.8119 - val_loss: 0.8875 - val_accuracy: 0.7340
Epoch 15/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.5360 - accuracy: 0.8272 - val_loss: 0.7529 - val_accuracy: 0.7660
Epoch 16/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.4950 - accuracy: 0.8394 - val_loss: 0.8427 - val_accuracy: 0.7457
Epoch 17/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.4548 - accuracy: 0.8504 - val_loss: 1.0679 - val_accuracy: 0.6867
Epoch 18/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.4241 - accuracy: 0.8612 - val_loss: 0.9413 - val_accuracy: 0.7193
Epoch 19/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.3995 - accuracy: 0.8709 - val_loss: 0.6902 - val_accuracy: 0.7943
Epoch 20/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.3783 - accuracy: 0.8769 - val_loss: 0.8582 - val_accuracy: 0.7583
Epoch 21/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.3481 - accuracy: 0.8847 - val_loss: 0.8370 - val_accuracy: 0.7620
Epoch 22/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.3185 - accuracy: 0.8945 - val_loss: 0.8169 - val_accuracy: 0.7690
Epoch 23/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.3103 - accuracy: 0.8995 - val_loss: 0.8569 - val_accuracy: 0.7697
Epoch 24/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.2829 - accuracy: 0.9072 - val_loss: 0.6503 - val_accuracy: 0.8153
Epoch 25/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.2699 - accuracy: 0.9121 - val_loss: 0.9422 - val_accuracy: 0.7607
Epoch 26/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.2569 - accuracy: 0.9171 - val_loss: 0.9949 - val_accuracy: 0.7530
Epoch 27/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.2476 - accuracy: 0.9186 - val_loss: 1.2068 - val_accuracy: 0.6940
Epoch 28/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.2361 - accuracy: 0.9232 - val_loss: 0.7116 - val_accuracy: 0.8143
Epoch 29/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.2239 - accuracy: 0.9254 - val_loss: 1.0246 - val_accuracy: 0.7457
Epoch 30/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.2123 - accuracy: 0.9288 - val_loss: 0.9247 - val_accuracy: 0.7640
Epoch 31/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.2012 - accuracy: 0.9343 - val_loss: 0.7154 - val_accuracy: 0.8157
Epoch 32/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.1897 - accuracy: 0.9379 - val_loss: 0.6897 - val_accuracy: 0.8143
Epoch 33/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1866 - accuracy: 0.9391 - val_loss: 0.6013 - val_accuracy: 0.8423
Epoch 34/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.1739 - accuracy: 0.9418 - val_loss: 0.7491 - val_accuracy: 0.8140
Epoch 35/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1769 - accuracy: 0.9430 - val_loss: 0.6189 - val_accuracy: 0.8403
Epoch 36/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.1632 - accuracy: 0.9468 - val_loss: 0.6661 - val_accuracy: 0.8353
Epoch 37/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1533 - accuracy: 0.9505 - val_loss: 0.7066 - val_accuracy: 0.8187
Epoch 38/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1617 - accuracy: 0.9465 - val_loss: 0.9854 - val_accuracy: 0.7673
Epoch 39/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1439 - accuracy: 0.9529 - val_loss: 0.5418 - val_accuracy: 0.8653
Epoch 40/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.1377 - accuracy: 0.9571 - val_loss: 0.8073 - val_accuracy: 0.7963
Epoch 41/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1336 - accuracy: 0.9571 - val_loss: 0.5265 - val_accuracy: 0.8630
Epoch 42/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.1315 - accuracy: 0.9569 - val_loss: 0.7069 - val_accuracy: 0.8367
Epoch 43/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1316 - accuracy: 0.9539 - val_loss: 0.5694 - val_accuracy: 0.8573
Epoch 44/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1233 - accuracy: 0.9616 - val_loss: 0.5464 - val_accuracy: 0.8640
Epoch 45/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1210 - accuracy: 0.9603 - val_loss: 0.6435 - val_accuracy: 0.8477
Epoch 46/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.1181 - accuracy: 0.9608 - val_loss: 0.5334 - val_accuracy: 0.8727
Epoch 47/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.1115 - accuracy: 0.9631 - val_loss: 0.5075 - val_accuracy: 0.8743
Epoch 48/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.1113 - accuracy: 0.9627 - val_loss: 1.1768 - val_accuracy: 0.7530
Epoch 49/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.1039 - accuracy: 0.9653 - val_loss: 0.5916 - val_accuracy: 0.8600
Epoch 50/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1090 - accuracy: 0.9650 - val_loss: 0.6513 - val_accuracy: 0.8540
Epoch 51/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.1018 - accuracy: 0.9694 - val_loss: 0.6781 - val_accuracy: 0.8490
Epoch 52/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0951 - accuracy: 0.9694 - val_loss: 0.8663 - val_accuracy: 0.8163
Epoch 53/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.1022 - accuracy: 0.9674 - val_loss: 0.7769 - val_accuracy: 0.8253
Epoch 54/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0949 - accuracy: 0.9678 - val_loss: 0.6695 - val_accuracy: 0.8543
Epoch 55/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0955 - accuracy: 0.9684 - val_loss: 0.6884 - val_accuracy: 0.8423
Epoch 56/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0858 - accuracy: 0.9725 - val_loss: 0.6409 - val_accuracy: 0.8577
Epoch 57/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0862 - accuracy: 0.9730 - val_loss: 0.8465 - val_accuracy: 0.8243
Epoch 58/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0846 - accuracy: 0.9727 - val_loss: 0.7029 - val_accuracy: 0.8497
Epoch 59/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0907 - accuracy: 0.9701 - val_loss: 0.8971 - val_accuracy: 0.8163
Epoch 60/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0906 - accuracy: 0.9712 - val_loss: 0.8174 - val_accuracy: 0.8300
Epoch 61/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0722 - accuracy: 0.9768 - val_loss: 0.6890 - val_accuracy: 0.8563
Epoch 62/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0779 - accuracy: 0.9747 - val_loss: 0.5491 - val_accuracy: 0.8767
Epoch 63/200
1433/1433 [==============================] - 34s 23ms/step - loss: 0.0811 - accuracy: 0.9733 - val_loss: 0.4884 - val_accuracy: 0.8937
Epoch 64/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0787 - accuracy: 0.9756 - val_loss: 0.6160 - val_accuracy: 0.8703
Epoch 65/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0753 - accuracy: 0.9743 - val_loss: 0.6959 - val_accuracy: 0.8487
Epoch 66/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0721 - accuracy: 0.9768 - val_loss: 0.6585 - val_accuracy: 0.8603
Epoch 67/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0761 - accuracy: 0.9758 - val_loss: 0.6024 - val_accuracy: 0.8650
Epoch 68/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0746 - accuracy: 0.9758 - val_loss: 1.0620 - val_accuracy: 0.7800
Epoch 69/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0801 - accuracy: 0.9749 - val_loss: 0.7605 - val_accuracy: 0.8347
Epoch 70/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0695 - accuracy: 0.9783 - val_loss: 0.8550 - val_accuracy: 0.8157
Epoch 71/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0654 - accuracy: 0.9790 - val_loss: 0.7106 - val_accuracy: 0.8480
Epoch 72/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0711 - accuracy: 0.9782 - val_loss: 0.7312 - val_accuracy: 0.8400
Epoch 73/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0671 - accuracy: 0.9793 - val_loss: 0.6192 - val_accuracy: 0.8677
Epoch 74/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0726 - accuracy: 0.9754 - val_loss: 0.5522 - val_accuracy: 0.8810
Epoch 75/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0701 - accuracy: 0.9771 - val_loss: 0.4732 - val_accuracy: 0.8907
Epoch 76/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0522 - accuracy: 0.9832 - val_loss: 0.6256 - val_accuracy: 0.8730
Epoch 77/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0629 - accuracy: 0.9800 - val_loss: 0.8407 - val_accuracy: 0.8393
Epoch 78/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0674 - accuracy: 0.9786 - val_loss: 0.5644 - val_accuracy: 0.8810
Epoch 79/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0659 - accuracy: 0.9802 - val_loss: 0.5602 - val_accuracy: 0.8693
Epoch 80/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0637 - accuracy: 0.9799 - val_loss: 0.5984 - val_accuracy: 0.8727
Epoch 81/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0527 - accuracy: 0.9818 - val_loss: 0.7291 - val_accuracy: 0.8587
Epoch 82/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0614 - accuracy: 0.9801 - val_loss: 0.7524 - val_accuracy: 0.8477
Epoch 83/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0596 - accuracy: 0.9814 - val_loss: 0.5875 - val_accuracy: 0.8797
Epoch 84/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0589 - accuracy: 0.9816 - val_loss: 0.6595 - val_accuracy: 0.8617
Epoch 85/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0554 - accuracy: 0.9819 - val_loss: 0.6387 - val_accuracy: 0.8723
Epoch 86/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0558 - accuracy: 0.9815 - val_loss: 0.6076 - val_accuracy: 0.8703
Epoch 87/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0529 - accuracy: 0.9831 - val_loss: 0.5955 - val_accuracy: 0.8743
Epoch 88/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0588 - accuracy: 0.9813 - val_loss: 0.5570 - val_accuracy: 0.8807
Epoch 89/200
1433/1433 [==============================] - 37s 26ms/step - loss: 0.0550 - accuracy: 0.9821 - val_loss: 0.5704 - val_accuracy: 0.8757
Epoch 90/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0586 - accuracy: 0.9807 - val_loss: 0.7071 - val_accuracy: 0.8467
Epoch 91/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0540 - accuracy: 0.9830 - val_loss: 0.5540 - val_accuracy: 0.8903
Epoch 92/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0499 - accuracy: 0.9844 - val_loss: 0.5972 - val_accuracy: 0.8793
Epoch 93/200
1433/1433 [==============================] - 37s 26ms/step - loss: 0.0559 - accuracy: 0.9821 - val_loss: 0.6795 - val_accuracy: 0.8550
Epoch 94/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0516 - accuracy: 0.9823 - val_loss: 0.7390 - val_accuracy: 0.8533
Epoch 95/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0502 - accuracy: 0.9849 - val_loss: 0.8823 - val_accuracy: 0.8290
Epoch 96/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0531 - accuracy: 0.9836 - val_loss: 0.5660 - val_accuracy: 0.8843
Epoch 97/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0557 - accuracy: 0.9823 - val_loss: 0.5465 - val_accuracy: 0.8867
Epoch 98/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0532 - accuracy: 0.9835 - val_loss: 0.5160 - val_accuracy: 0.8827
Epoch 99/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0472 - accuracy: 0.9841 - val_loss: 0.7948 - val_accuracy: 0.8510
Epoch 100/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0520 - accuracy: 0.9838 - val_loss: 0.8297 - val_accuracy: 0.8367
Epoch 101/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0564 - accuracy: 0.9807 - val_loss: 0.5948 - val_accuracy: 0.8803
Epoch 102/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0485 - accuracy: 0.9839 - val_loss: 0.5113 - val_accuracy: 0.8997
Epoch 103/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0504 - accuracy: 0.9848 - val_loss: 0.8628 - val_accuracy: 0.8417
Epoch 104/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0440 - accuracy: 0.9850 - val_loss: 0.5867 - val_accuracy: 0.8907
Epoch 105/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0494 - accuracy: 0.9851 - val_loss: 0.6070 - val_accuracy: 0.8693
Epoch 106/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0446 - accuracy: 0.9849 - val_loss: 0.7344 - val_accuracy: 0.8653
Epoch 107/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0463 - accuracy: 0.9853 - val_loss: 0.6184 - val_accuracy: 0.8780
Epoch 108/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0447 - accuracy: 0.9870 - val_loss: 0.5188 - val_accuracy: 0.8943
Epoch 109/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0458 - accuracy: 0.9863 - val_loss: 0.7709 - val_accuracy: 0.8553
Epoch 110/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0453 - accuracy: 0.9859 - val_loss: 0.6104 - val_accuracy: 0.8810
Epoch 111/200
1433/1433 [==============================] - 37s 26ms/step - loss: 0.0397 - accuracy: 0.9883 - val_loss: 0.5451 - val_accuracy: 0.8933
Epoch 112/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0449 - accuracy: 0.9862 - val_loss: 0.5823 - val_accuracy: 0.8903
Epoch 113/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0414 - accuracy: 0.9864 - val_loss: 0.5157 - val_accuracy: 0.8997
Epoch 114/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0451 - accuracy: 0.9862 - val_loss: 0.5491 - val_accuracy: 0.8947
Epoch 115/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0388 - accuracy: 0.9873 - val_loss: 0.6394 - val_accuracy: 0.8687
Epoch 116/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0440 - accuracy: 0.9872 - val_loss: 0.6828 - val_accuracy: 0.8713
Epoch 117/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0367 - accuracy: 0.9876 - val_loss: 0.5146 - val_accuracy: 0.9020
Epoch 118/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0452 - accuracy: 0.9860 - val_loss: 0.5144 - val_accuracy: 0.8980
Epoch 119/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0425 - accuracy: 0.9865 - val_loss: 0.4453 - val_accuracy: 0.9097
Epoch 120/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0447 - accuracy: 0.9855 - val_loss: 0.7529 - val_accuracy: 0.8483
Epoch 121/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0363 - accuracy: 0.9883 - val_loss: 0.6651 - val_accuracy: 0.8753
Epoch 122/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0383 - accuracy: 0.9883 - val_loss: 0.5657 - val_accuracy: 0.8953
Epoch 123/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0356 - accuracy: 0.9883 - val_loss: 0.5691 - val_accuracy: 0.8937
Epoch 124/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0449 - accuracy: 0.9860 - val_loss: 0.6695 - val_accuracy: 0.8663
Epoch 125/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0373 - accuracy: 0.9879 - val_loss: 0.6499 - val_accuracy: 0.8753
Epoch 126/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0402 - accuracy: 0.9874 - val_loss: 0.5500 - val_accuracy: 0.8927
Epoch 127/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0407 - accuracy: 0.9870 - val_loss: 0.5819 - val_accuracy: 0.8870
Epoch 128/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0354 - accuracy: 0.9881 - val_loss: 0.8850 - val_accuracy: 0.8473
Epoch 129/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0353 - accuracy: 0.9889 - val_loss: 0.4668 - val_accuracy: 0.9037
Epoch 130/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0317 - accuracy: 0.9895 - val_loss: 0.7203 - val_accuracy: 0.8637
Epoch 131/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0395 - accuracy: 0.9862 - val_loss: 0.7806 - val_accuracy: 0.8560
Epoch 132/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0384 - accuracy: 0.9885 - val_loss: 0.5996 - val_accuracy: 0.8883
Epoch 133/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0393 - accuracy: 0.9882 - val_loss: 0.6547 - val_accuracy: 0.8730
Epoch 134/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0327 - accuracy: 0.9892 - val_loss: 0.6682 - val_accuracy: 0.8777
Epoch 135/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0371 - accuracy: 0.9883 - val_loss: 0.7259 - val_accuracy: 0.8693
Epoch 136/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0414 - accuracy: 0.9859 - val_loss: 0.6067 - val_accuracy: 0.8807
Epoch 137/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0320 - accuracy: 0.9911 - val_loss: 0.5681 - val_accuracy: 0.8890
Epoch 138/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0407 - accuracy: 0.9876 - val_loss: 0.8422 - val_accuracy: 0.8557
Epoch 139/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0382 - accuracy: 0.9885 - val_loss: 0.6168 - val_accuracy: 0.8830
Epoch 140/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0441 - accuracy: 0.9866 - val_loss: 0.6326 - val_accuracy: 0.8787
Epoch 141/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0328 - accuracy: 0.9892 - val_loss: 0.6369 - val_accuracy: 0.8857
Epoch 142/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0304 - accuracy: 0.9907 - val_loss: 0.6997 - val_accuracy: 0.8680
Epoch 143/200
1433/1433 [==============================] - 34s 24ms/step - loss: 0.0373 - accuracy: 0.9890 - val_loss: 0.7815 - val_accuracy: 0.8553
Epoch 144/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0343 - accuracy: 0.9890 - val_loss: 0.4972 - val_accuracy: 0.8983
Epoch 145/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0332 - accuracy: 0.9889 - val_loss: 0.7471 - val_accuracy: 0.8690
Epoch 146/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0323 - accuracy: 0.9898 - val_loss: 0.6144 - val_accuracy: 0.8950
Epoch 147/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0347 - accuracy: 0.9886 - val_loss: 0.6190 - val_accuracy: 0.8843
Epoch 148/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0318 - accuracy: 0.9892 - val_loss: 0.8268 - val_accuracy: 0.8437
Epoch 149/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0313 - accuracy: 0.9895 - val_loss: 0.6810 - val_accuracy: 0.8697
Epoch 150/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0319 - accuracy: 0.9898 - val_loss: 0.5981 - val_accuracy: 0.8933
Epoch 151/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0354 - accuracy: 0.9888 - val_loss: 0.8229 - val_accuracy: 0.8533
Epoch 152/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0305 - accuracy: 0.9910 - val_loss: 0.8058 - val_accuracy: 0.8630
Epoch 153/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0364 - accuracy: 0.9894 - val_loss: 0.7149 - val_accuracy: 0.8663
Epoch 154/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0310 - accuracy: 0.9908 - val_loss: 0.5486 - val_accuracy: 0.9037
Epoch 155/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0344 - accuracy: 0.9890 - val_loss: 0.6404 - val_accuracy: 0.8867
Epoch 156/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0332 - accuracy: 0.9892 - val_loss: 0.6145 - val_accuracy: 0.8790
Epoch 157/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0357 - accuracy: 0.9883 - val_loss: 0.5764 - val_accuracy: 0.8880
Epoch 158/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0311 - accuracy: 0.9901 - val_loss: 0.7001 - val_accuracy: 0.8713
Epoch 159/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0347 - accuracy: 0.9899 - val_loss: 0.5486 - val_accuracy: 0.8927
Epoch 160/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0270 - accuracy: 0.9909 - val_loss: 0.4568 - val_accuracy: 0.9143
Epoch 161/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0325 - accuracy: 0.9898 - val_loss: 0.5939 - val_accuracy: 0.8867
Epoch 162/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0317 - accuracy: 0.9892 - val_loss: 0.7557 - val_accuracy: 0.8687
Epoch 163/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0288 - accuracy: 0.9906 - val_loss: 0.5325 - val_accuracy: 0.8977
Epoch 164/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0314 - accuracy: 0.9894 - val_loss: 0.6314 - val_accuracy: 0.8820
Epoch 165/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0318 - accuracy: 0.9890 - val_loss: 0.5714 - val_accuracy: 0.8903
Epoch 166/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0279 - accuracy: 0.9909 - val_loss: 0.5789 - val_accuracy: 0.8950
Epoch 167/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0319 - accuracy: 0.9897 - val_loss: 0.5151 - val_accuracy: 0.8987
Epoch 168/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0362 - accuracy: 0.9890 - val_loss: 0.6323 - val_accuracy: 0.8793
Epoch 169/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0261 - accuracy: 0.9918 - val_loss: 0.5896 - val_accuracy: 0.9000
Epoch 170/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0321 - accuracy: 0.9899 - val_loss: 0.7249 - val_accuracy: 0.8750
Epoch 171/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0308 - accuracy: 0.9904 - val_loss: 0.5601 - val_accuracy: 0.9020
Epoch 172/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0283 - accuracy: 0.9917 - val_loss: 1.0358 - val_accuracy: 0.8373
Epoch 173/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0293 - accuracy: 0.9919 - val_loss: 0.6793 - val_accuracy: 0.8873
Epoch 174/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0318 - accuracy: 0.9899 - val_loss: 0.6796 - val_accuracy: 0.8793
Epoch 175/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0300 - accuracy: 0.9900 - val_loss: 0.6466 - val_accuracy: 0.8890
Epoch 176/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0271 - accuracy: 0.9911 - val_loss: 0.5532 - val_accuracy: 0.9023
Epoch 177/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0275 - accuracy: 0.9916 - val_loss: 0.7893 - val_accuracy: 0.8687
Epoch 178/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0268 - accuracy: 0.9910 - val_loss: 0.6094 - val_accuracy: 0.8893
Epoch 179/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0337 - accuracy: 0.9894 - val_loss: 0.7218 - val_accuracy: 0.8700
Epoch 180/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0255 - accuracy: 0.9919 - val_loss: 0.8094 - val_accuracy: 0.8627
Epoch 181/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0325 - accuracy: 0.9900 - val_loss: 0.7417 - val_accuracy: 0.8770
Epoch 182/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0302 - accuracy: 0.9911 - val_loss: 0.6308 - val_accuracy: 0.8827
Epoch 183/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0281 - accuracy: 0.9909 - val_loss: 0.5958 - val_accuracy: 0.8963
Epoch 184/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0337 - accuracy: 0.9912 - val_loss: 0.7662 - val_accuracy: 0.8660
Epoch 185/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0278 - accuracy: 0.9915 - val_loss: 0.5143 - val_accuracy: 0.8980
Epoch 186/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0255 - accuracy: 0.9918 - val_loss: 0.5656 - val_accuracy: 0.8993
Epoch 187/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0280 - accuracy: 0.9914 - val_loss: 0.8227 - val_accuracy: 0.8617
Epoch 188/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0312 - accuracy: 0.9909 - val_loss: 0.7492 - val_accuracy: 0.8583
Epoch 189/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0288 - accuracy: 0.9904 - val_loss: 0.5576 - val_accuracy: 0.8927
Epoch 190/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0303 - accuracy: 0.9903 - val_loss: 0.6551 - val_accuracy: 0.8847
Epoch 191/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0275 - accuracy: 0.9913 - val_loss: 0.5430 - val_accuracy: 0.9000
Epoch 192/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0265 - accuracy: 0.9911 - val_loss: 0.5965 - val_accuracy: 0.8990
Epoch 193/200
1433/1433 [==============================] - 35s 25ms/step - loss: 0.0287 - accuracy: 0.9920 - val_loss: 0.6010 - val_accuracy: 0.8867
Epoch 194/200
1433/1433 [==============================] - 35s 24ms/step - loss: 0.0243 - accuracy: 0.9923 - val_loss: 0.5916 - val_accuracy: 0.8930
Epoch 195/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0293 - accuracy: 0.9906 - val_loss: 0.8563 - val_accuracy: 0.8563
Epoch 196/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0258 - accuracy: 0.9927 - val_loss: 0.9959 - val_accuracy: 0.8517
Epoch 197/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0321 - accuracy: 0.9905 - val_loss: 0.6024 - val_accuracy: 0.8950
Epoch 198/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0237 - accuracy: 0.9929 - val_loss: 0.6999 - val_accuracy: 0.8800
Epoch 199/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0237 - accuracy: 0.9924 - val_loss: 0.7260 - val_accuracy: 0.8790
Epoch 200/200
1433/1433 [==============================] - 36s 25ms/step - loss: 0.0254 - accuracy: 0.9923 - val_loss: 0.6716 - val_accuracy: 0.8910

Conv2D_31_aug.summary()

Model: "Conv2D_31_Augmentation"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalised_data (Sequential  (None, 31, 31, 1)        0         
 )                                                               
                                                                 
 conv2d (Conv2D)             (None, 31, 31, 32)        320       
                                                                 
 max_pooling2d (MaxPooling2D  (None, 15, 15, 32)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 15, 15, 64)        18496     
                                                                 
 conv2d_2 (Conv2D)           (None, 15, 15, 128)       73856     
                                                                 
 conv2d_3 (Conv2D)           (None, 15, 15, 128)       147584    
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 7, 7, 128)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 6272)              0         
                                                                 
 dropout (Dropout)           (None, 6272)              0         
                                                                 
 dense (Dense)               (None, 512)               3211776   
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_1 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_2 (Dense)             (None, 15)                3855      
                                                                 
=================================================================
Total params: 3,587,215
Trainable params: 3,587,215
Non-trainable params: 0
_________________________________________________________________

plot_learning_curve(Conv2D_31_aug_history.history)

Observations

• From the graph, the validation loss and accuracy seems to be diverging away from the training curve of the augmented data which is worse compared to the validation curve of the unaugmented data.

Conv2D_31_aug.evaluate(test_data_31.batch(10))

300/300 [==============================] - 2s 6ms/step - loss: 0.6455 - accuracy: 0.8927

[0.6455073356628418, 0.8926666378974915]

Observations

• The test accuracy shows that the accuracy was quite low, which shows that augmentation of the data does not acutally improve the 31 x 31 image.

CNN Augmented Balance Version 2 (31 x 31)

• Increased units of Conv2D and kernel size

# Try for 31 x 31 images
tf.keras.backend.clear_session()

Conv2D_31_aug_V2 = Sequential(name="Conv2D_31_Augmentation_V2",
    layers = [
        normalised_data,
        Conv2D(64, (5, 5),input_shape=(31, 31 ,1), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Conv2D(128, (3, 3), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Conv2D(256, (3, 3), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.6),

        Dense(128, activation='relu'),
        Dropout(0.6),

        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.0001)
Conv2D_31_aug_V2.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_31_aug_V2.build(input_shape=(None, 31, 31, 1))

Conv2D_31_aug_V2_history = Conv2D_31_aug_V2.fit(
    train_31V2.batch(10), 
    epochs=200,
    validation_data=val_data_31.batch(10)
    )

Epoch 1/200
1433/1433 [==============================] - 8s 5ms/step - loss: 2.6791 - accuracy: 0.0912 - val_loss: 2.5895 - val_accuracy: 0.1237
Epoch 2/200
1433/1433 [==============================] - 8s 5ms/step - loss: 2.4913 - accuracy: 0.1742 - val_loss: 2.3339 - val_accuracy: 0.2207
Epoch 3/200
1433/1433 [==============================] - 7s 5ms/step - loss: 2.3014 - accuracy: 0.2469 - val_loss: 2.2397 - val_accuracy: 0.2697
Epoch 4/200
1433/1433 [==============================] - 7s 5ms/step - loss: 2.1385 - accuracy: 0.3050 - val_loss: 2.2808 - val_accuracy: 0.2903
Epoch 5/200
1433/1433 [==============================] - 7s 5ms/step - loss: 2.0127 - accuracy: 0.3464 - val_loss: 1.9834 - val_accuracy: 0.3743
Epoch 6/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.9103 - accuracy: 0.3881 - val_loss: 1.8005 - val_accuracy: 0.4210
Epoch 7/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.8167 - accuracy: 0.4143 - val_loss: 1.7014 - val_accuracy: 0.4497
Epoch 8/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.7186 - accuracy: 0.4482 - val_loss: 1.6432 - val_accuracy: 0.4710
Epoch 9/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.6310 - accuracy: 0.4730 - val_loss: 1.6081 - val_accuracy: 0.4773
Epoch 10/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.5572 - accuracy: 0.5042 - val_loss: 1.5235 - val_accuracy: 0.5090
Epoch 11/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.4899 - accuracy: 0.5184 - val_loss: 1.3843 - val_accuracy: 0.5637
Epoch 12/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.4241 - accuracy: 0.5401 - val_loss: 1.3903 - val_accuracy: 0.5520
Epoch 13/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.3637 - accuracy: 0.5603 - val_loss: 1.2711 - val_accuracy: 0.6013
Epoch 14/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.3142 - accuracy: 0.5777 - val_loss: 1.4041 - val_accuracy: 0.5463
Epoch 15/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.2655 - accuracy: 0.5936 - val_loss: 1.2650 - val_accuracy: 0.5917
Epoch 16/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.2211 - accuracy: 0.6111 - val_loss: 1.1810 - val_accuracy: 0.6197
Epoch 17/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.1738 - accuracy: 0.6227 - val_loss: 1.0544 - val_accuracy: 0.6577
Epoch 18/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.1248 - accuracy: 0.6373 - val_loss: 1.2063 - val_accuracy: 0.6060
Epoch 19/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.0903 - accuracy: 0.6505 - val_loss: 1.0751 - val_accuracy: 0.6560
Epoch 20/200
1433/1433 [==============================] - 8s 5ms/step - loss: 1.0748 - accuracy: 0.6606 - val_loss: 0.9407 - val_accuracy: 0.6980
Epoch 21/200
1433/1433 [==============================] - 8s 5ms/step - loss: 1.0312 - accuracy: 0.6688 - val_loss: 0.9679 - val_accuracy: 0.6850
Epoch 22/200
1433/1433 [==============================] - 7s 5ms/step - loss: 1.0013 - accuracy: 0.6762 - val_loss: 0.9426 - val_accuracy: 0.6983
Epoch 23/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.9778 - accuracy: 0.6884 - val_loss: 1.0482 - val_accuracy: 0.6547
Epoch 24/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.9486 - accuracy: 0.6948 - val_loss: 1.0221 - val_accuracy: 0.6667
Epoch 25/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.9175 - accuracy: 0.7006 - val_loss: 0.9335 - val_accuracy: 0.6953
Epoch 26/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.9001 - accuracy: 0.7054 - val_loss: 0.9928 - val_accuracy: 0.6727
Epoch 27/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.8691 - accuracy: 0.7180 - val_loss: 0.8718 - val_accuracy: 0.7177
Epoch 28/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.8381 - accuracy: 0.7268 - val_loss: 1.0691 - val_accuracy: 0.6557
Epoch 29/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.8365 - accuracy: 0.7296 - val_loss: 0.9452 - val_accuracy: 0.6880
Epoch 30/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.7951 - accuracy: 0.7416 - val_loss: 0.7991 - val_accuracy: 0.7377
Epoch 31/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.7901 - accuracy: 0.7471 - val_loss: 0.9303 - val_accuracy: 0.6860
Epoch 32/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.7784 - accuracy: 0.7447 - val_loss: 1.0020 - val_accuracy: 0.6730
Epoch 33/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.7612 - accuracy: 0.7525 - val_loss: 0.7367 - val_accuracy: 0.7640
Epoch 34/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.7461 - accuracy: 0.7598 - val_loss: 0.7524 - val_accuracy: 0.7593
Epoch 35/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.7214 - accuracy: 0.7642 - val_loss: 0.7628 - val_accuracy: 0.7523
Epoch 36/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.7062 - accuracy: 0.7703 - val_loss: 0.8996 - val_accuracy: 0.7117
Epoch 37/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.6905 - accuracy: 0.7735 - val_loss: 0.8903 - val_accuracy: 0.7120
Epoch 38/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.6818 - accuracy: 0.7710 - val_loss: 0.7878 - val_accuracy: 0.7453
Epoch 39/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.6741 - accuracy: 0.7803 - val_loss: 0.9392 - val_accuracy: 0.6997
Epoch 40/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.6606 - accuracy: 0.7830 - val_loss: 0.8917 - val_accuracy: 0.7180
Epoch 41/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.6440 - accuracy: 0.7892 - val_loss: 0.9791 - val_accuracy: 0.6860
Epoch 42/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.6372 - accuracy: 0.7878 - val_loss: 0.7749 - val_accuracy: 0.7447
Epoch 43/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.6234 - accuracy: 0.7990 - val_loss: 0.8067 - val_accuracy: 0.7333
Epoch 44/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.6061 - accuracy: 0.8002 - val_loss: 0.7429 - val_accuracy: 0.7640
Epoch 45/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5966 - accuracy: 0.8055 - val_loss: 0.7973 - val_accuracy: 0.7450
Epoch 46/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5870 - accuracy: 0.8046 - val_loss: 0.7810 - val_accuracy: 0.7510
Epoch 47/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5620 - accuracy: 0.8126 - val_loss: 0.7173 - val_accuracy: 0.7657
Epoch 48/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5606 - accuracy: 0.8151 - val_loss: 0.6779 - val_accuracy: 0.7813
Epoch 49/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5561 - accuracy: 0.8158 - val_loss: 0.7087 - val_accuracy: 0.7703
Epoch 50/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5567 - accuracy: 0.8176 - val_loss: 0.8302 - val_accuracy: 0.7357
Epoch 51/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5347 - accuracy: 0.8258 - val_loss: 0.7501 - val_accuracy: 0.7597
Epoch 52/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5278 - accuracy: 0.8246 - val_loss: 0.7928 - val_accuracy: 0.7427
Epoch 53/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5224 - accuracy: 0.8258 - val_loss: 0.5999 - val_accuracy: 0.8097
Epoch 54/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5091 - accuracy: 0.8329 - val_loss: 0.6171 - val_accuracy: 0.8090
Epoch 55/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.5000 - accuracy: 0.8375 - val_loss: 0.5570 - val_accuracy: 0.8233
Epoch 56/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4926 - accuracy: 0.8375 - val_loss: 0.7984 - val_accuracy: 0.7467
Epoch 57/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4899 - accuracy: 0.8364 - val_loss: 0.5411 - val_accuracy: 0.8290
Epoch 58/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4818 - accuracy: 0.8401 - val_loss: 0.8514 - val_accuracy: 0.7287
Epoch 59/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4789 - accuracy: 0.8405 - val_loss: 0.5856 - val_accuracy: 0.8147
Epoch 60/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4777 - accuracy: 0.8436 - val_loss: 0.6444 - val_accuracy: 0.7990
Epoch 61/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4505 - accuracy: 0.8521 - val_loss: 0.5932 - val_accuracy: 0.8153
Epoch 62/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4684 - accuracy: 0.8469 - val_loss: 0.6726 - val_accuracy: 0.7933
Epoch 63/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4453 - accuracy: 0.8544 - val_loss: 0.6544 - val_accuracy: 0.8010
Epoch 64/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4346 - accuracy: 0.8573 - val_loss: 0.6847 - val_accuracy: 0.7867
Epoch 65/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4380 - accuracy: 0.8554 - val_loss: 0.8865 - val_accuracy: 0.7333
Epoch 66/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4326 - accuracy: 0.8569 - val_loss: 0.5072 - val_accuracy: 0.8480
Epoch 67/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4248 - accuracy: 0.8595 - val_loss: 0.5374 - val_accuracy: 0.8323
Epoch 68/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4187 - accuracy: 0.8620 - val_loss: 0.5009 - val_accuracy: 0.8497
Epoch 69/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.4056 - accuracy: 0.8637 - val_loss: 0.6612 - val_accuracy: 0.7947
Epoch 70/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3884 - accuracy: 0.8713 - val_loss: 0.6277 - val_accuracy: 0.8027
Epoch 71/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.4028 - accuracy: 0.8664 - val_loss: 0.5611 - val_accuracy: 0.8270
Epoch 72/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.3823 - accuracy: 0.8727 - val_loss: 0.7229 - val_accuracy: 0.7820
Epoch 73/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3834 - accuracy: 0.8753 - val_loss: 0.5752 - val_accuracy: 0.8250
Epoch 74/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3879 - accuracy: 0.8715 - val_loss: 0.5246 - val_accuracy: 0.8397
Epoch 75/200
1433/1433 [==============================] - 8s 5ms/step - loss: 0.3809 - accuracy: 0.8748 - val_loss: 0.5696 - val_accuracy: 0.8247
Epoch 76/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3830 - accuracy: 0.8721 - val_loss: 0.7646 - val_accuracy: 0.7723
Epoch 77/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3658 - accuracy: 0.8777 - val_loss: 0.8070 - val_accuracy: 0.7647
Epoch 78/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3552 - accuracy: 0.8820 - val_loss: 0.5926 - val_accuracy: 0.8227
Epoch 79/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3631 - accuracy: 0.8780 - val_loss: 0.4679 - val_accuracy: 0.8570
Epoch 80/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3656 - accuracy: 0.8767 - val_loss: 0.5556 - val_accuracy: 0.8310
Epoch 81/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3365 - accuracy: 0.8875 - val_loss: 0.6435 - val_accuracy: 0.8080
Epoch 82/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3421 - accuracy: 0.8878 - val_loss: 0.5636 - val_accuracy: 0.8287
Epoch 83/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3484 - accuracy: 0.8861 - val_loss: 0.7130 - val_accuracy: 0.7890
Epoch 84/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3182 - accuracy: 0.8917 - val_loss: 0.5582 - val_accuracy: 0.8353
Epoch 85/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3179 - accuracy: 0.8939 - val_loss: 0.5605 - val_accuracy: 0.8313
Epoch 86/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3300 - accuracy: 0.8902 - val_loss: 0.5467 - val_accuracy: 0.8353
Epoch 87/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3246 - accuracy: 0.8899 - val_loss: 0.5499 - val_accuracy: 0.8333
Epoch 88/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3151 - accuracy: 0.8958 - val_loss: 0.5479 - val_accuracy: 0.8300
Epoch 89/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3081 - accuracy: 0.8965 - val_loss: 0.6928 - val_accuracy: 0.7990
Epoch 90/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3158 - accuracy: 0.8931 - val_loss: 0.4723 - val_accuracy: 0.8547
Epoch 91/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.3043 - accuracy: 0.8985 - val_loss: 0.4710 - val_accuracy: 0.8600
Epoch 92/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.2929 - accuracy: 0.9035 - val_loss: 0.6752 - val_accuracy: 0.8113
Epoch 93/200
1433/1433 [==============================] - 7s 5ms/step - loss: 0.2968 - accuracy: 0.9008 - val_loss: 0.6695 - val_accuracy: 0.8120
Epoch 94/200
1433/1433 [==============================] - 4s 3ms/step - loss: 0.2931 - accuracy: 0.9026 - val_loss: 0.5312 - val_accuracy: 0.8457
Epoch 95/200
1433/1433 [==============================] - 4s 3ms/step - loss: 0.2917 - accuracy: 0.9032 - val_loss: 0.4859 - val_accuracy: 0.8647
Epoch 96/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2768 - accuracy: 0.9072 - val_loss: 0.5740 - val_accuracy: 0.8400
Epoch 97/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2792 - accuracy: 0.9063 - val_loss: 0.6210 - val_accuracy: 0.8323
Epoch 98/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2761 - accuracy: 0.9074 - val_loss: 0.5076 - val_accuracy: 0.8540
Epoch 99/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2770 - accuracy: 0.9062 - val_loss: 0.5369 - val_accuracy: 0.8493
Epoch 100/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2822 - accuracy: 0.9069 - val_loss: 0.4793 - val_accuracy: 0.8607
Epoch 101/200
1433/1433 [==============================] - 4s 3ms/step - loss: 0.2706 - accuracy: 0.9082 - val_loss: 0.4849 - val_accuracy: 0.8717
Epoch 102/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2603 - accuracy: 0.9144 - val_loss: 0.5982 - val_accuracy: 0.8390
Epoch 103/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2708 - accuracy: 0.9094 - val_loss: 0.6743 - val_accuracy: 0.8153
Epoch 104/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2626 - accuracy: 0.9129 - val_loss: 0.6330 - val_accuracy: 0.8350
Epoch 105/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2572 - accuracy: 0.9164 - val_loss: 0.4963 - val_accuracy: 0.8630
Epoch 106/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2585 - accuracy: 0.9109 - val_loss: 0.4893 - val_accuracy: 0.8583
Epoch 107/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2513 - accuracy: 0.9147 - val_loss: 0.6215 - val_accuracy: 0.8277
Epoch 108/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2614 - accuracy: 0.9125 - val_loss: 0.6032 - val_accuracy: 0.8347
Epoch 109/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2507 - accuracy: 0.9162 - val_loss: 0.4525 - val_accuracy: 0.8710
Epoch 110/200
1433/1433 [==============================] - 4s 3ms/step - loss: 0.2569 - accuracy: 0.9141 - val_loss: 0.6098 - val_accuracy: 0.8380
Epoch 111/200
1433/1433 [==============================] - 4s 3ms/step - loss: 0.2386 - accuracy: 0.9199 - val_loss: 0.5560 - val_accuracy: 0.8510
Epoch 112/200
1433/1433 [==============================] - 4s 3ms/step - loss: 0.2570 - accuracy: 0.9151 - val_loss: 0.4653 - val_accuracy: 0.8707
Epoch 113/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2391 - accuracy: 0.9222 - val_loss: 0.6283 - val_accuracy: 0.8317
Epoch 114/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2453 - accuracy: 0.9174 - val_loss: 0.5077 - val_accuracy: 0.8580
Epoch 115/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2327 - accuracy: 0.9228 - val_loss: 0.5130 - val_accuracy: 0.8693
Epoch 116/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2337 - accuracy: 0.9218 - val_loss: 0.5588 - val_accuracy: 0.8470
Epoch 117/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2311 - accuracy: 0.9252 - val_loss: 0.3941 - val_accuracy: 0.8957
Epoch 118/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2285 - accuracy: 0.9253 - val_loss: 0.5262 - val_accuracy: 0.8590
Epoch 119/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2259 - accuracy: 0.9252 - val_loss: 0.6257 - val_accuracy: 0.8410
Epoch 120/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2226 - accuracy: 0.9264 - val_loss: 0.4776 - val_accuracy: 0.8687
Epoch 121/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2284 - accuracy: 0.9227 - val_loss: 0.5358 - val_accuracy: 0.8633
Epoch 122/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2208 - accuracy: 0.9271 - val_loss: 0.5382 - val_accuracy: 0.8617
Epoch 123/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2218 - accuracy: 0.9270 - val_loss: 0.4852 - val_accuracy: 0.8737
Epoch 124/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2214 - accuracy: 0.9261 - val_loss: 0.6586 - val_accuracy: 0.8340
Epoch 125/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2140 - accuracy: 0.9280 - val_loss: 0.4971 - val_accuracy: 0.8727
Epoch 126/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2174 - accuracy: 0.9287 - val_loss: 0.4675 - val_accuracy: 0.8750
Epoch 127/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2121 - accuracy: 0.9288 - val_loss: 0.4678 - val_accuracy: 0.8763
Epoch 128/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2108 - accuracy: 0.9296 - val_loss: 0.6128 - val_accuracy: 0.8457
Epoch 129/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2142 - accuracy: 0.9338 - val_loss: 0.5618 - val_accuracy: 0.8440
Epoch 130/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2138 - accuracy: 0.9291 - val_loss: 0.5590 - val_accuracy: 0.8607
Epoch 131/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2027 - accuracy: 0.9330 - val_loss: 0.5682 - val_accuracy: 0.8533
Epoch 132/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2106 - accuracy: 0.9309 - val_loss: 0.5032 - val_accuracy: 0.8617
Epoch 133/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1949 - accuracy: 0.9345 - val_loss: 0.5959 - val_accuracy: 0.8507
Epoch 134/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1963 - accuracy: 0.9359 - val_loss: 0.5468 - val_accuracy: 0.8613
Epoch 135/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1903 - accuracy: 0.9365 - val_loss: 0.5889 - val_accuracy: 0.8513
Epoch 136/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.2020 - accuracy: 0.9343 - val_loss: 0.4622 - val_accuracy: 0.8850
Epoch 137/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1922 - accuracy: 0.9387 - val_loss: 0.4519 - val_accuracy: 0.8820
Epoch 138/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1921 - accuracy: 0.9356 - val_loss: 0.4518 - val_accuracy: 0.8830
Epoch 139/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1950 - accuracy: 0.9351 - val_loss: 0.5489 - val_accuracy: 0.8700
Epoch 140/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1846 - accuracy: 0.9382 - val_loss: 0.5581 - val_accuracy: 0.8627
Epoch 141/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1871 - accuracy: 0.9374 - val_loss: 0.3891 - val_accuracy: 0.8953
Epoch 142/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1756 - accuracy: 0.9416 - val_loss: 0.5656 - val_accuracy: 0.8617
Epoch 143/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1810 - accuracy: 0.9418 - val_loss: 0.7128 - val_accuracy: 0.8260
Epoch 144/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1767 - accuracy: 0.9432 - val_loss: 0.4951 - val_accuracy: 0.8777
Epoch 145/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1839 - accuracy: 0.9393 - val_loss: 0.6520 - val_accuracy: 0.8337
Epoch 146/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1722 - accuracy: 0.9416 - val_loss: 0.6001 - val_accuracy: 0.8570
Epoch 147/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1750 - accuracy: 0.9430 - val_loss: 0.6430 - val_accuracy: 0.8463
Epoch 148/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1727 - accuracy: 0.9423 - val_loss: 0.5493 - val_accuracy: 0.8630
Epoch 149/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1703 - accuracy: 0.9446 - val_loss: 0.4862 - val_accuracy: 0.8753
Epoch 150/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1810 - accuracy: 0.9401 - val_loss: 0.5553 - val_accuracy: 0.8590
Epoch 151/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1661 - accuracy: 0.9453 - val_loss: 0.6713 - val_accuracy: 0.8443
Epoch 152/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1699 - accuracy: 0.9444 - val_loss: 0.6109 - val_accuracy: 0.8520
Epoch 153/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1713 - accuracy: 0.9446 - val_loss: 0.4501 - val_accuracy: 0.8947
Epoch 154/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1702 - accuracy: 0.9425 - val_loss: 0.5737 - val_accuracy: 0.8640
Epoch 155/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1576 - accuracy: 0.9476 - val_loss: 0.5685 - val_accuracy: 0.8670
Epoch 156/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1620 - accuracy: 0.9446 - val_loss: 0.4745 - val_accuracy: 0.8833
Epoch 157/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1588 - accuracy: 0.9488 - val_loss: 0.5186 - val_accuracy: 0.8820
Epoch 158/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1580 - accuracy: 0.9485 - val_loss: 0.5039 - val_accuracy: 0.8790
Epoch 159/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1648 - accuracy: 0.9472 - val_loss: 0.5480 - val_accuracy: 0.8690
Epoch 160/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1596 - accuracy: 0.9488 - val_loss: 0.5604 - val_accuracy: 0.8647
Epoch 161/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1588 - accuracy: 0.9490 - val_loss: 0.4173 - val_accuracy: 0.9003
Epoch 162/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1643 - accuracy: 0.9464 - val_loss: 0.4577 - val_accuracy: 0.8903
Epoch 163/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1542 - accuracy: 0.9497 - val_loss: 0.5198 - val_accuracy: 0.8763
Epoch 164/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1525 - accuracy: 0.9499 - val_loss: 0.5690 - val_accuracy: 0.8663
Epoch 165/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1489 - accuracy: 0.9498 - val_loss: 0.5142 - val_accuracy: 0.8790
Epoch 166/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1527 - accuracy: 0.9489 - val_loss: 0.4444 - val_accuracy: 0.8910
Epoch 167/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1538 - accuracy: 0.9491 - val_loss: 0.6927 - val_accuracy: 0.8497
Epoch 168/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1489 - accuracy: 0.9515 - val_loss: 0.5954 - val_accuracy: 0.8550
Epoch 169/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1371 - accuracy: 0.9520 - val_loss: 0.5334 - val_accuracy: 0.8777
Epoch 170/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1410 - accuracy: 0.9509 - val_loss: 0.6308 - val_accuracy: 0.8447
Epoch 171/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1508 - accuracy: 0.9516 - val_loss: 0.4128 - val_accuracy: 0.8950
Epoch 172/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1508 - accuracy: 0.9488 - val_loss: 0.4964 - val_accuracy: 0.8820
Epoch 173/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1461 - accuracy: 0.9509 - val_loss: 0.5515 - val_accuracy: 0.8650
Epoch 174/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1388 - accuracy: 0.9537 - val_loss: 0.5874 - val_accuracy: 0.8613
Epoch 175/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1381 - accuracy: 0.9532 - val_loss: 0.5599 - val_accuracy: 0.8687
Epoch 176/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1384 - accuracy: 0.9539 - val_loss: 0.5102 - val_accuracy: 0.8783
Epoch 177/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1455 - accuracy: 0.9523 - val_loss: 0.5347 - val_accuracy: 0.8813
Epoch 178/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1435 - accuracy: 0.9529 - val_loss: 0.4426 - val_accuracy: 0.8933
Epoch 179/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1448 - accuracy: 0.9521 - val_loss: 0.5747 - val_accuracy: 0.8713
Epoch 180/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1343 - accuracy: 0.9568 - val_loss: 0.5781 - val_accuracy: 0.8647
Epoch 181/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1455 - accuracy: 0.9528 - val_loss: 0.4960 - val_accuracy: 0.8793
Epoch 182/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1397 - accuracy: 0.9540 - val_loss: 0.4969 - val_accuracy: 0.8797
Epoch 183/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1413 - accuracy: 0.9532 - val_loss: 0.5016 - val_accuracy: 0.8807
Epoch 184/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1308 - accuracy: 0.9569 - val_loss: 0.4421 - val_accuracy: 0.8973
Epoch 185/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1338 - accuracy: 0.9553 - val_loss: 0.4678 - val_accuracy: 0.8887
Epoch 186/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1295 - accuracy: 0.9565 - val_loss: 0.4277 - val_accuracy: 0.9050
Epoch 187/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1297 - accuracy: 0.9580 - val_loss: 0.5563 - val_accuracy: 0.8757
Epoch 188/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1247 - accuracy: 0.9585 - val_loss: 0.4414 - val_accuracy: 0.9017
Epoch 189/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1323 - accuracy: 0.9557 - val_loss: 0.6075 - val_accuracy: 0.8640
Epoch 190/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1380 - accuracy: 0.9551 - val_loss: 0.4264 - val_accuracy: 0.8993
Epoch 191/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.1188 - accuracy: 0.9626 - val_loss: 0.5232 - val_accuracy: 0.8833
Epoch 192/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1279 - accuracy: 0.9595 - val_loss: 0.5264 - val_accuracy: 0.8877
Epoch 193/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.1249 - accuracy: 0.9587 - val_loss: 0.4957 - val_accuracy: 0.8830
Epoch 194/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1262 - accuracy: 0.9588 - val_loss: 0.5238 - val_accuracy: 0.8800
Epoch 195/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1168 - accuracy: 0.9636 - val_loss: 0.5397 - val_accuracy: 0.8807
Epoch 196/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1218 - accuracy: 0.9599 - val_loss: 0.4939 - val_accuracy: 0.8917
Epoch 197/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1245 - accuracy: 0.9580 - val_loss: 0.5823 - val_accuracy: 0.8753
Epoch 198/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1205 - accuracy: 0.9608 - val_loss: 0.5289 - val_accuracy: 0.8803
Epoch 199/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1239 - accuracy: 0.9590 - val_loss: 0.5534 - val_accuracy: 0.8787
Epoch 200/200
1433/1433 [==============================] - 5s 3ms/step - loss: 0.1200 - accuracy: 0.9608 - val_loss: 0.4880 - val_accuracy: 0.8950

Conv2D_31_aug_V2.summary()

Model: "Conv2D_31_Augmentation_V2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalised_data (Sequential  (None, 31, 31, 1)        0         
 )                                                               
                                                                 
 conv2d (Conv2D)             (None, 31, 31, 64)        1664      
                                                                 
 max_pooling2d (MaxPooling2D  (None, 15, 15, 64)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 15, 15, 128)       73856     
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 7, 7, 128)        0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 7, 7, 256)         295168    
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 3, 3, 256)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 2304)              0         
                                                                 
 dropout (Dropout)           (None, 2304)              0         
                                                                 
 dense (Dense)               (None, 128)               295040    
                                                                 
 dropout_1 (Dropout)         (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 15)                1935      
                                                                 
=================================================================
Total params: 667,663
Trainable params: 667,663
Non-trainable params: 0
_________________________________________________________________

plot_learning_curve(Conv2D_31_aug_V2_history.history)

Observations

• The loss curve shows constant fluctuations in the validation loss which is generally higher than the training loss.
• The accuracy curve also shows constant fluctuations in the validation accuracy which is generally lower than the training accuracy.

Conv2D_31_aug_V2.evaluate(test_data_31.batch(10))

300/300 [==============================] - 1s 3ms/step - loss: 0.4920 - accuracy: 0.8943

[0.49203088879585266, 0.8943333625793457]

Observations

• The accuracy is slightly better than the model before but still not as good as the unaugmented one.

4) Model Improvement

128 x 128 images

Time-based Decay

• Used to adjust the learning rate during training by decreasing it overtime
• Starts with a relatively large learning rate and gradually reduce it as training progress.
• Beneficial for optimization and help the model to converge more efficiently
• We will make use of learning rate scheduler to make this work

$$ \text{{learning\_rate}} = \text{{learning\_rate}} \times \left(1 + \frac{k \cdot \text{{initial learning\_rate}}}{{\text{{epochs}} + 1}}\right)^{-1} $$

def time_based_decay(epoch, lr):
    initial_lr = 0.00025
    #declare number of steps
    k = 0.1
    #Avoid division error
    decay = initial_lr /(epoch+1)
    lr *= (1. / (1. + k*decay * initial_lr))
    return lr

lr_schedule = LearningRateScheduler(time_based_decay, verbose=1)

# Try for 128 x 128 images
tf.keras.backend.clear_session()

Conv2D_128V1_aug_lr = Sequential(name="Conv2D_128V1_Augment_LearningRate",
    layers = [
        normalised_data,
        Conv2D(64, (5, 5), activation='relu', padding='same', input_shape=(128, 128, 1), strides=(4, 4)),
        MaxPooling2D((2, 2)),
        Conv2D(128, (3, 3), activation='relu', padding='same', strides=(1, 1)),
        MaxPooling2D((2, 2)),
        Conv2D(128, (3, 3), activation='relu', padding='same', strides=(1, 1)),
        MaxPooling2D((2, 2)),
        Conv2D(256, (3, 3), activation='relu', padding='same', strides=(1, 1)),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.5),

        Dense(1024, activation='relu'),
        Dropout(0.5),
        Dense(128, activation = 'relu'),
        Dropout(0.5),
        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.00025)
Conv2D_128V1_aug_lr.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_128V1_aug_lr.build(input_shape=(None, 128, 128, 1))

Conv2D_128V1_aug_lr_history = Conv2D_128V1_aug_lr.fit(
    train_128V2.batch(10), 
    epochs=100,
    validation_data=val_data_128.batch(10),
    callbacks=[lr_schedule]
    )

Epoch 1: LearningRateScheduler setting learning rate to 0.00025000001031186275.
Epoch 1/100
1433/1433 [==============================] - 9s 6ms/step - loss: 2.6363 - accuracy: 0.0958 - val_loss: 2.3595 - val_accuracy: 0.2137 - lr: 2.5000e-04

Epoch 2: LearningRateScheduler setting learning rate to 0.00025000001109311276.
Epoch 2/100
1433/1433 [==============================] - 8s 6ms/step - loss: 2.2485 - accuracy: 0.2393 - val_loss: 1.8425 - val_accuracy: 0.3780 - lr: 2.5000e-04

Epoch 3: LearningRateScheduler setting learning rate to 0.0002500000113535295.
Epoch 3/100
1433/1433 [==============================] - 8s 6ms/step - loss: 1.8423 - accuracy: 0.3862 - val_loss: 1.3211 - val_accuracy: 0.5687 - lr: 2.5000e-04

Epoch 4: LearningRateScheduler setting learning rate to 0.00025000001148373785.
Epoch 4/100
1433/1433 [==============================] - 8s 6ms/step - loss: 1.4413 - accuracy: 0.5315 - val_loss: 1.0554 - val_accuracy: 0.6553 - lr: 2.5000e-04

Epoch 5: LearningRateScheduler setting learning rate to 0.0002500000115618628.
Epoch 5/100
1433/1433 [==============================] - 8s 6ms/step - loss: 1.1397 - accuracy: 0.6369 - val_loss: 0.9110 - val_accuracy: 0.7157 - lr: 2.5000e-04

Epoch 6: LearningRateScheduler setting learning rate to 0.00025000001161394614.
Epoch 6/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.9151 - accuracy: 0.7116 - val_loss: 0.7029 - val_accuracy: 0.7890 - lr: 2.5000e-04

Epoch 7: LearningRateScheduler setting learning rate to 0.00025000001165114853.
Epoch 7/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.7500 - accuracy: 0.7651 - val_loss: 0.7759 - val_accuracy: 0.7657 - lr: 2.5000e-04

Epoch 8: LearningRateScheduler setting learning rate to 0.0002500000116790503.
Epoch 8/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.6322 - accuracy: 0.8013 - val_loss: 0.5151 - val_accuracy: 0.8407 - lr: 2.5000e-04

Epoch 9: LearningRateScheduler setting learning rate to 0.0002500000117007517.
Epoch 9/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.5447 - accuracy: 0.8297 - val_loss: 0.5238 - val_accuracy: 0.8450 - lr: 2.5000e-04

Epoch 10: LearningRateScheduler setting learning rate to 0.0002500000117181128.
Epoch 10/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.4582 - accuracy: 0.8554 - val_loss: 0.4419 - val_accuracy: 0.8553 - lr: 2.5000e-04

Epoch 11: LearningRateScheduler setting learning rate to 0.0002500000117323174.
Epoch 11/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4191 - accuracy: 0.8678 - val_loss: 0.3217 - val_accuracy: 0.9020 - lr: 2.5000e-04

Epoch 12: LearningRateScheduler setting learning rate to 0.0002500000117441545.
Epoch 12/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3637 - accuracy: 0.8871 - val_loss: 0.3700 - val_accuracy: 0.8917 - lr: 2.5000e-04

Epoch 13: LearningRateScheduler setting learning rate to 0.0002500000117541705.
Epoch 13/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.3154 - accuracy: 0.9025 - val_loss: 0.4717 - val_accuracy: 0.8713 - lr: 2.5000e-04

Epoch 14: LearningRateScheduler setting learning rate to 0.00025000001176275565.
Epoch 14/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.2843 - accuracy: 0.9125 - val_loss: 0.4592 - val_accuracy: 0.8667 - lr: 2.5000e-04

Epoch 15: LearningRateScheduler setting learning rate to 0.00025000001177019615.
Epoch 15/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.2669 - accuracy: 0.9162 - val_loss: 0.3014 - val_accuracy: 0.9180 - lr: 2.5000e-04

Epoch 16: LearningRateScheduler setting learning rate to 0.00025000001177670657.
Epoch 16/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.2310 - accuracy: 0.9294 - val_loss: 0.4103 - val_accuracy: 0.8923 - lr: 2.5000e-04

Epoch 17: LearningRateScheduler setting learning rate to 0.00025000001178245105.
Epoch 17/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.2067 - accuracy: 0.9370 - val_loss: 0.4424 - val_accuracy: 0.8910 - lr: 2.5000e-04

Epoch 18: LearningRateScheduler setting learning rate to 0.00025000001178755726.
Epoch 18/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1894 - accuracy: 0.9405 - val_loss: 0.4405 - val_accuracy: 0.8927 - lr: 2.5000e-04

Epoch 19: LearningRateScheduler setting learning rate to 0.000250000011792126.
Epoch 19/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.1804 - accuracy: 0.9462 - val_loss: 0.3550 - val_accuracy: 0.9083 - lr: 2.5000e-04

Epoch 20: LearningRateScheduler setting learning rate to 0.0002500000117962378.
Epoch 20/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.1787 - accuracy: 0.9462 - val_loss: 0.4428 - val_accuracy: 0.8970 - lr: 2.5000e-04

Epoch 21: LearningRateScheduler setting learning rate to 0.00025000001179995804.
Epoch 21/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.1535 - accuracy: 0.9529 - val_loss: 0.3616 - val_accuracy: 0.9080 - lr: 2.5000e-04

Epoch 22: LearningRateScheduler setting learning rate to 0.0002500000118033401.
Epoch 22/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.1620 - accuracy: 0.9502 - val_loss: 0.4059 - val_accuracy: 0.8920 - lr: 2.5000e-04

Epoch 23: LearningRateScheduler setting learning rate to 0.0002500000118064281.
Epoch 23/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.1373 - accuracy: 0.9579 - val_loss: 0.2841 - val_accuracy: 0.9210 - lr: 2.5000e-04

Epoch 24: LearningRateScheduler setting learning rate to 0.00025000001180925865.
Epoch 24/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1269 - accuracy: 0.9609 - val_loss: 0.2652 - val_accuracy: 0.9313 - lr: 2.5000e-04

Epoch 25: LearningRateScheduler setting learning rate to 0.0002500000118118628.
Epoch 25/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.1412 - accuracy: 0.9601 - val_loss: 0.4236 - val_accuracy: 0.9083 - lr: 2.5000e-04

Epoch 26: LearningRateScheduler setting learning rate to 0.0002500000118142667.
Epoch 26/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.1172 - accuracy: 0.9643 - val_loss: 0.2711 - val_accuracy: 0.9333 - lr: 2.5000e-04

Epoch 27: LearningRateScheduler setting learning rate to 0.00025000001181649245.
Epoch 27/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.1227 - accuracy: 0.9638 - val_loss: 0.3402 - val_accuracy: 0.9243 - lr: 2.5000e-04

Epoch 28: LearningRateScheduler setting learning rate to 0.00025000001181855927.
Epoch 28/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.1126 - accuracy: 0.9664 - val_loss: 0.3173 - val_accuracy: 0.9237 - lr: 2.5000e-04

Epoch 29: LearningRateScheduler setting learning rate to 0.0002500000118204835.
Epoch 29/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1103 - accuracy: 0.9684 - val_loss: 0.2720 - val_accuracy: 0.9333 - lr: 2.5000e-04

Epoch 30: LearningRateScheduler setting learning rate to 0.0002500000118222795.
Epoch 30/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1082 - accuracy: 0.9694 - val_loss: 0.5094 - val_accuracy: 0.8947 - lr: 2.5000e-04

Epoch 31: LearningRateScheduler setting learning rate to 0.00025000001182395957.
Epoch 31/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.1049 - accuracy: 0.9698 - val_loss: 0.3909 - val_accuracy: 0.9127 - lr: 2.5000e-04

Epoch 32: LearningRateScheduler setting learning rate to 0.0002500000118255347.
Epoch 32/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1036 - accuracy: 0.9701 - val_loss: 0.3156 - val_accuracy: 0.9283 - lr: 2.5000e-04

Epoch 33: LearningRateScheduler setting learning rate to 0.00025000001182701436.
Epoch 33/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0902 - accuracy: 0.9738 - val_loss: 0.3705 - val_accuracy: 0.9220 - lr: 2.5000e-04

Epoch 34: LearningRateScheduler setting learning rate to 0.0002500000118284069.
Epoch 34/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0894 - accuracy: 0.9732 - val_loss: 0.2808 - val_accuracy: 0.9323 - lr: 2.5000e-04

Epoch 35: LearningRateScheduler setting learning rate to 0.00025000001182972.
Epoch 35/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0846 - accuracy: 0.9765 - val_loss: 0.3797 - val_accuracy: 0.9173 - lr: 2.5000e-04

Epoch 36: LearningRateScheduler setting learning rate to 0.00025000001183096004.
Epoch 36/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0829 - accuracy: 0.9761 - val_loss: 0.2852 - val_accuracy: 0.9420 - lr: 2.5000e-04

Epoch 37: LearningRateScheduler setting learning rate to 0.0002500000118321331.
Epoch 37/100
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0913 - accuracy: 0.9730 - val_loss: 0.2475 - val_accuracy: 0.9443 - lr: 2.5000e-04

Epoch 38: LearningRateScheduler setting learning rate to 0.0002500000118332444.
Epoch 38/100
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0738 - accuracy: 0.9795 - val_loss: 0.3859 - val_accuracy: 0.9310 - lr: 2.5000e-04

Epoch 39: LearningRateScheduler setting learning rate to 0.0002500000118342987.
Epoch 39/100
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0869 - accuracy: 0.9750 - val_loss: 0.4548 - val_accuracy: 0.9153 - lr: 2.5000e-04

Epoch 40: LearningRateScheduler setting learning rate to 0.0002500000118353003.
Epoch 40/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0757 - accuracy: 0.9791 - val_loss: 0.3054 - val_accuracy: 0.9380 - lr: 2.5000e-04

Epoch 41: LearningRateScheduler setting learning rate to 0.00025000001183625307.
Epoch 41/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0811 - accuracy: 0.9779 - val_loss: 0.3473 - val_accuracy: 0.9277 - lr: 2.5000e-04

Epoch 42: LearningRateScheduler setting learning rate to 0.00025000001183716043.
Epoch 42/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0757 - accuracy: 0.9788 - val_loss: 0.4628 - val_accuracy: 0.9153 - lr: 2.5000e-04

Epoch 43: LearningRateScheduler setting learning rate to 0.00025000001183802563.
Epoch 43/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0716 - accuracy: 0.9801 - val_loss: 0.5132 - val_accuracy: 0.9057 - lr: 2.5000e-04

Epoch 44: LearningRateScheduler setting learning rate to 0.00025000001183885146.
Epoch 44/100
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0785 - accuracy: 0.9787 - val_loss: 0.2932 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 45: LearningRateScheduler setting learning rate to 0.0002500000118396406.
Epoch 45/100
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0707 - accuracy: 0.9795 - val_loss: 0.2573 - val_accuracy: 0.9387 - lr: 2.5000e-04

Epoch 46: LearningRateScheduler setting learning rate to 0.0002500000118403954.
Epoch 46/100
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0648 - accuracy: 0.9820 - val_loss: 0.2514 - val_accuracy: 0.9470 - lr: 2.5000e-04

Epoch 47: LearningRateScheduler setting learning rate to 0.00025000001184111815.
Epoch 47/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0871 - accuracy: 0.9767 - val_loss: 0.2593 - val_accuracy: 0.9453 - lr: 2.5000e-04

Epoch 48: LearningRateScheduler setting learning rate to 0.00025000001184181074.
Epoch 48/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0591 - accuracy: 0.9825 - val_loss: 0.3162 - val_accuracy: 0.9413 - lr: 2.5000e-04

Epoch 49: LearningRateScheduler setting learning rate to 0.00025000001184247503.
Epoch 49/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0727 - accuracy: 0.9800 - val_loss: 0.2685 - val_accuracy: 0.9473 - lr: 2.5000e-04

Epoch 50: LearningRateScheduler setting learning rate to 0.0002500000118431128.
Epoch 50/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0478 - accuracy: 0.9856 - val_loss: 0.3740 - val_accuracy: 0.9213 - lr: 2.5000e-04

Epoch 51: LearningRateScheduler setting learning rate to 0.00025000001184372555.
Epoch 51/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0722 - accuracy: 0.9812 - val_loss: 0.2508 - val_accuracy: 0.9443 - lr: 2.5000e-04

Epoch 52: LearningRateScheduler setting learning rate to 0.00025000001184431476.
Epoch 52/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0648 - accuracy: 0.9823 - val_loss: 0.3245 - val_accuracy: 0.9323 - lr: 2.5000e-04

Epoch 53: LearningRateScheduler setting learning rate to 0.0002500000118448817.
Epoch 53/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0700 - accuracy: 0.9818 - val_loss: 0.2597 - val_accuracy: 0.9450 - lr: 2.5000e-04

Epoch 54: LearningRateScheduler setting learning rate to 0.00025000001184542764.
Epoch 54/100
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0584 - accuracy: 0.9845 - val_loss: 0.3528 - val_accuracy: 0.9307 - lr: 2.5000e-04

Epoch 55: LearningRateScheduler setting learning rate to 0.0002500000118459537.
Epoch 55/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0555 - accuracy: 0.9838 - val_loss: 0.3001 - val_accuracy: 0.9393 - lr: 2.5000e-04

Epoch 56: LearningRateScheduler setting learning rate to 0.00025000001184646105.
Epoch 56/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0575 - accuracy: 0.9849 - val_loss: 0.3065 - val_accuracy: 0.9347 - lr: 2.5000e-04

Epoch 57: LearningRateScheduler setting learning rate to 0.0002500000118469505.
Epoch 57/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0715 - accuracy: 0.9820 - val_loss: 0.5653 - val_accuracy: 0.8947 - lr: 2.5000e-04

Epoch 58: LearningRateScheduler setting learning rate to 0.00025000001184742317.
Epoch 58/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0472 - accuracy: 0.9861 - val_loss: 0.4155 - val_accuracy: 0.9283 - lr: 2.5000e-04

Epoch 59: LearningRateScheduler setting learning rate to 0.0002500000118478798.
Epoch 59/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0620 - accuracy: 0.9828 - val_loss: 0.3766 - val_accuracy: 0.9307 - lr: 2.5000e-04

Epoch 60: LearningRateScheduler setting learning rate to 0.00025000001184832116.
Epoch 60/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0555 - accuracy: 0.9850 - val_loss: 0.4964 - val_accuracy: 0.9130 - lr: 2.5000e-04

Epoch 61: LearningRateScheduler setting learning rate to 0.0002500000118487481.
Epoch 61/100
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0530 - accuracy: 0.9856 - val_loss: 0.3497 - val_accuracy: 0.9380 - lr: 2.5000e-04

Epoch 62: LearningRateScheduler setting learning rate to 0.0002500000118491612.
Epoch 62/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0641 - accuracy: 0.9821 - val_loss: 0.3113 - val_accuracy: 0.9360 - lr: 2.5000e-04

Epoch 63: LearningRateScheduler setting learning rate to 0.0002500000118495612.
Epoch 63/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0451 - accuracy: 0.9881 - val_loss: 0.3578 - val_accuracy: 0.9357 - lr: 2.5000e-04

Epoch 64: LearningRateScheduler setting learning rate to 0.00025000001184994876.
Epoch 64/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0678 - accuracy: 0.9831 - val_loss: 0.3112 - val_accuracy: 0.9433 - lr: 2.5000e-04

Epoch 65: LearningRateScheduler setting learning rate to 0.0002500000118503244.
Epoch 65/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0598 - accuracy: 0.9841 - val_loss: 0.3793 - val_accuracy: 0.9310 - lr: 2.5000e-04

Epoch 66: LearningRateScheduler setting learning rate to 0.0002500000118506886.
Epoch 66/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0473 - accuracy: 0.9872 - val_loss: 0.3178 - val_accuracy: 0.9403 - lr: 2.5000e-04

Epoch 67: LearningRateScheduler setting learning rate to 0.0002500000118510419.
Epoch 67/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0516 - accuracy: 0.9867 - val_loss: 0.6868 - val_accuracy: 0.8967 - lr: 2.5000e-04

Epoch 68: LearningRateScheduler setting learning rate to 0.0002500000118513849.
Epoch 68/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0562 - accuracy: 0.9851 - val_loss: 0.3507 - val_accuracy: 0.9353 - lr: 2.5000e-04

Epoch 69: LearningRateScheduler setting learning rate to 0.0002500000118517179.
Epoch 69/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0607 - accuracy: 0.9840 - val_loss: 0.3233 - val_accuracy: 0.9430 - lr: 2.5000e-04

Epoch 70: LearningRateScheduler setting learning rate to 0.0002500000118520414.
Epoch 70/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0476 - accuracy: 0.9870 - val_loss: 0.3193 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 71: LearningRateScheduler setting learning rate to 0.00025000001185235574.
Epoch 71/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0518 - accuracy: 0.9849 - val_loss: 0.3405 - val_accuracy: 0.9390 - lr: 2.5000e-04

Epoch 72: LearningRateScheduler setting learning rate to 0.00025000001185266144.
Epoch 72/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0474 - accuracy: 0.9875 - val_loss: 0.3088 - val_accuracy: 0.9470 - lr: 2.5000e-04

Epoch 73: LearningRateScheduler setting learning rate to 0.0002500000118529587.
Epoch 73/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0502 - accuracy: 0.9867 - val_loss: 0.3606 - val_accuracy: 0.9387 - lr: 2.5000e-04

Epoch 74: LearningRateScheduler setting learning rate to 0.00025000001185324793.
Epoch 74/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0614 - accuracy: 0.9851 - val_loss: 0.3393 - val_accuracy: 0.9393 - lr: 2.5000e-04

Epoch 75: LearningRateScheduler setting learning rate to 0.0002500000118535295.
Epoch 75/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0584 - accuracy: 0.9843 - val_loss: 0.2700 - val_accuracy: 0.9447 - lr: 2.5000e-04

Epoch 76: LearningRateScheduler setting learning rate to 0.0002500000118538036.
Epoch 76/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0369 - accuracy: 0.9890 - val_loss: 0.5829 - val_accuracy: 0.9183 - lr: 2.5000e-04

Epoch 77: LearningRateScheduler setting learning rate to 0.00025000001185407063.
Epoch 77/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0530 - accuracy: 0.9864 - val_loss: 0.3432 - val_accuracy: 0.9433 - lr: 2.5000e-04

Epoch 78: LearningRateScheduler setting learning rate to 0.0002500000118543308.
Epoch 78/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0526 - accuracy: 0.9865 - val_loss: 0.5694 - val_accuracy: 0.9113 - lr: 2.5000e-04

Epoch 79: LearningRateScheduler setting learning rate to 0.0002500000118545844.
Epoch 79/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0497 - accuracy: 0.9857 - val_loss: 0.3256 - val_accuracy: 0.9437 - lr: 2.5000e-04

Epoch 80: LearningRateScheduler setting learning rate to 0.0002500000118548316.
Epoch 80/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0564 - accuracy: 0.9862 - val_loss: 0.2922 - val_accuracy: 0.9467 - lr: 2.5000e-04

Epoch 81: LearningRateScheduler setting learning rate to 0.0002500000118550727.
Epoch 81/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0450 - accuracy: 0.9883 - val_loss: 0.2771 - val_accuracy: 0.9480 - lr: 2.5000e-04

Epoch 82: LearningRateScheduler setting learning rate to 0.000250000011855308.
Epoch 82/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0506 - accuracy: 0.9877 - val_loss: 0.2938 - val_accuracy: 0.9440 - lr: 2.5000e-04

Epoch 83: LearningRateScheduler setting learning rate to 0.00025000001185553755.
Epoch 83/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0565 - accuracy: 0.9850 - val_loss: 0.4004 - val_accuracy: 0.9337 - lr: 2.5000e-04

Epoch 84: LearningRateScheduler setting learning rate to 0.00025000001185576166.
Epoch 84/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0462 - accuracy: 0.9880 - val_loss: 0.3538 - val_accuracy: 0.9423 - lr: 2.5000e-04

Epoch 85: LearningRateScheduler setting learning rate to 0.00025000001185598045.
Epoch 85/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0451 - accuracy: 0.9890 - val_loss: 0.2893 - val_accuracy: 0.9477 - lr: 2.5000e-04

Epoch 86: LearningRateScheduler setting learning rate to 0.00025000001185619425.
Epoch 86/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0450 - accuracy: 0.9880 - val_loss: 0.2762 - val_accuracy: 0.9490 - lr: 2.5000e-04

Epoch 87: LearningRateScheduler setting learning rate to 0.00025000001185640307.
Epoch 87/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0461 - accuracy: 0.9883 - val_loss: 0.3107 - val_accuracy: 0.9453 - lr: 2.5000e-04

Epoch 88: LearningRateScheduler setting learning rate to 0.0002500000118566071.
Epoch 88/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0508 - accuracy: 0.9859 - val_loss: 0.3201 - val_accuracy: 0.9497 - lr: 2.5000e-04

Epoch 89: LearningRateScheduler setting learning rate to 0.00025000001185680667.
Epoch 89/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0614 - accuracy: 0.9858 - val_loss: 0.4596 - val_accuracy: 0.9183 - lr: 2.5000e-04

Epoch 90: LearningRateScheduler setting learning rate to 0.0002500000118570017.
Epoch 90/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0361 - accuracy: 0.9902 - val_loss: 0.4262 - val_accuracy: 0.9333 - lr: 2.5000e-04

Epoch 91: LearningRateScheduler setting learning rate to 0.0002500000118571925.
Epoch 91/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0482 - accuracy: 0.9888 - val_loss: 0.3794 - val_accuracy: 0.9343 - lr: 2.5000e-04

Epoch 92: LearningRateScheduler setting learning rate to 0.0002500000118573791.
Epoch 92/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0530 - accuracy: 0.9874 - val_loss: 0.3498 - val_accuracy: 0.9397 - lr: 2.5000e-04

Epoch 93: LearningRateScheduler setting learning rate to 0.00025000001185756176.
Epoch 93/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0437 - accuracy: 0.9889 - val_loss: 0.4223 - val_accuracy: 0.9297 - lr: 2.5000e-04

Epoch 94: LearningRateScheduler setting learning rate to 0.0002500000118577405.
Epoch 94/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0389 - accuracy: 0.9895 - val_loss: 0.4820 - val_accuracy: 0.9300 - lr: 2.5000e-04

Epoch 95: LearningRateScheduler setting learning rate to 0.0002500000118579155.
Epoch 95/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0533 - accuracy: 0.9884 - val_loss: 0.3726 - val_accuracy: 0.9353 - lr: 2.5000e-04

Epoch 96: LearningRateScheduler setting learning rate to 0.0002500000118580868.
Epoch 96/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0430 - accuracy: 0.9903 - val_loss: 0.3610 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 97: LearningRateScheduler setting learning rate to 0.0002500000118582546.
Epoch 97/100
1433/1433 [==============================] - 8s 5ms/step - loss: 0.0539 - accuracy: 0.9869 - val_loss: 0.3338 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 98: LearningRateScheduler setting learning rate to 0.000250000011858419.
Epoch 98/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0465 - accuracy: 0.9902 - val_loss: 0.3100 - val_accuracy: 0.9447 - lr: 2.5000e-04

Epoch 99: LearningRateScheduler setting learning rate to 0.00025000001185858.
Epoch 99/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0454 - accuracy: 0.9893 - val_loss: 0.5453 - val_accuracy: 0.9253 - lr: 2.5000e-04

Epoch 100: LearningRateScheduler setting learning rate to 0.00025000001185873785.
Epoch 100/100
1433/1433 [==============================] - 8s 6ms/step - loss: 0.0481 - accuracy: 0.9880 - val_loss: 0.3320 - val_accuracy: 0.9350 - lr: 2.5000e-04

Observations:

• Learning rate did not really change much but accuracy for validation is still relatively quite high, validation loss is also quite low

Conv2D_128V1_aug_lr.summary()

Model: "Conv2D_128V1_Augment_LearningRate"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalised_data (Sequential  (None, None, None, 1)    0         
 )                                                               
                                                                 
 conv2d (Conv2D)             (None, 32, 32, 64)        1664      
                                                                 
 max_pooling2d (MaxPooling2D  (None, 16, 16, 64)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 8, 8, 128)        0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 4, 4, 128)        0         
 2D)                                                             
                                                                 
 conv2d_3 (Conv2D)           (None, 4, 4, 256)         295168    
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 2, 2, 256)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 1024)              0         
                                                                 
 dropout (Dropout)           (None, 1024)              0         
                                                                 
 dense (Dense)               (None, 1024)              1049600   
                                                                 
 dropout_1 (Dropout)         (None, 1024)              0         
                                                                 
 dense_1 (Dense)             (None, 128)               131200    
                                                                 
 dropout_2 (Dropout)         (None, 128)               0         
                                                                 
 dense_2 (Dense)             (None, 15)                1935      
                                                                 
=================================================================
Total params: 1,701,007
Trainable params: 1,701,007
Non-trainable params: 0
_________________________________________________________________

plot_learning_curve(Conv2D_128V1_aug_lr_history.history)

Observations:

• Model seems to generate quite a low validation loss and a high validation accuracy, showing that the model architecture was quite good.

Conv2D_128V1_aug_lr.evaluate(test_data_128.batch(10))

300/300 [==============================] - 2s 6ms/step - loss: 0.3115 - accuracy: 0.9377

[0.31153255701065063, 0.937666654586792]

Observations:

• Test accuracy is quite high 0.94 which means this model architecture is quite good.

Keras Tuner

• An easy-to-use, scalable hyperparameter optimization framework that search the hyperparameter space to find the best set of hyperparameters

Optimizers

Adam

• An adaptive learning rate optimization algorithm
• Maintains a moving average of both the gradients and the second moments of the gradients
• Adapts the learning rate of each parameter individually

SGD (Stochastic Gradient Descent)

• Basic optimization algorithm which update the model parameters based on the negative gradient of the loss function with respect to each parameter.
• Each update is based on a random subset of the training data (mini batch)

RMSprop (Root Mean Square Propagation)

• An adaptive learning rate optimization algorithm
• Address some of the limitations of basic SGD by adapting the learning rates of each parameter
• Divides the learning rate for a weight by the moving average of the magnitude of recent gradients for that weight

def build_128_aug_model(hp):
    model = Sequential(
        layers = [
            normalised_data,
            Conv2D(64, (5, 5), activation='relu', padding='same', input_shape=(128, 128, 1), strides=(4, 4)),
            MaxPooling2D((2, 2)),
            Conv2D(128, (3, 3), activation='relu', padding='same', strides=(1, 1)),
            MaxPooling2D((2, 2)),
            Conv2D(128, (3, 3), activation='relu', padding='same', strides=(1, 1)),
            MaxPooling2D((2, 2)),
            Conv2D(256, (3, 3), activation='relu', padding='same', strides=(1, 1)),
            MaxPooling2D((2, 2)),

            Flatten(),
            Dropout(0.5),

            Dense(1024, activation='relu'),
            Dropout(0.5),
            Dense(128, activation = 'relu'),
            Dropout(0.5),
            Dense(num_classes, activation='softmax')
        ]
    )

    optimizer_name = hp.Choice('optimizer', values=['adam', 'sgd', 'rmsprop'])

    if optimizer_name == 'adam':
        optimizer = Adam(learning_rate = 0.00025)
    elif optimizer_name == 'sgd':
        optimizer = SGD(learning_rate = 0.00025, momentum = 0.9, nesterov= True)
    elif optimizer_name == 'rmsprop':
        optimizer = RMSprop(learning_rate = 0.000025, momentum= 0.9)

    model.build(input_shape=(None, 128, 128, 1))

    # Compile model
    model.compile(optimizer=optimizer, loss='sparse_categorical_crossentropy', metrics=['accuracy']) 
    return model

tuner = GridSearch(build_128_aug_model, objective='val_accuracy', max_trials=3)

#search the best parameter
best_params_128 = tuner.search(
    train_128V2.batch(10),
    epochs=200,
    validation_data=val_data_128.batch(10)
    )

Trial 3 Complete [00h 30m 15s]
val_accuracy: 0.859333336353302

Best val_accuracy So Far: 0.9490000009536743
Total elapsed time: 01h 29m 32s

# Get the best trial
best_trial_128 = tuner.oracle.get_best_trials(num_trials=1)[0]

# Access the best parameters
best_params_128 = best_trial_128.hyperparameters.values

print(f"Best Parameters: {best_params_128}")

Best Parameters: {'optimizer': 'adam'}

Observations

• Keras Tuner shows that Adam Optimizer is the best optimizer

Regularization - Reduces overfitting and improve generalisation performance of the model

• L1 Regularization:
  • Penalise the absolute value of the weights. Weights may be reduced to zero which is useful when compressing the model. $$ L1(\theta) = \lambda \sum_{i=1}^{n} |w_i| $$
• L2 Regularization:
  • Known as weight decay and usually better over L1. Forces the weights to decay towards zero. $$ L2(\theta) = \lambda \sum_{i=1}^{n} w_i^2 $$
• Dropout
  • Have been using this form of regularisation so far.
  • At every iteration, it randomly selects some nodes and removes them along with all of their incoming and outgoing connections as shown below.
• I have also increased the learning rate so that the model will learn and converge faster

# Try for 128 x 128 images
tf.keras.backend.clear_session()

Conv2D_128_improved_aug = Sequential(name="Conv2D_128_Final_Augmentation",
    layers = [
        normalised_data,
        Conv2D(64, (5, 5), activation='relu', padding='same', input_shape=(128, 128, 1), strides=(4, 4)),
        MaxPooling2D((2, 2)),
        Conv2D(128, (3, 3), activation='relu', padding='same', strides=(1, 1)),
        MaxPooling2D((2, 2)),
        Conv2D(128, (3, 3), activation='relu', padding='same', strides=(1, 1)),
        MaxPooling2D((2, 2)),
        Conv2D(256, (3, 3), activation='relu', padding='same', strides=(1, 1)),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.5),

        Dense(1024, activation='relu', kernel_regularizer=l2(0.001)),
        Dropout(0.5),
        Dense(128, activation = 'relu', kernel_regularizer=l2(0.001)),
        Dropout(0.5),
        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.0001)
Conv2D_128_improved_aug.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_128_improved_aug.build(input_shape=(None, 128, 128, 1))

Conv2D_128_improved_aug_history = Conv2D_128_improved_aug.fit(
    train_128V2.batch(10), 
    epochs=200,
    validation_data=val_data_128.batch(10)
    )

Epoch 1/200
1433/1433 [==============================] - 9s 6ms/step - loss: 3.2968 - accuracy: 0.0768 - val_loss: 2.8500 - val_accuracy: 0.1233
Epoch 2/200
1433/1433 [==============================] - 8s 6ms/step - loss: 2.6585 - accuracy: 0.1735 - val_loss: 2.2171 - val_accuracy: 0.3210
Epoch 3/200
1433/1433 [==============================] - 9s 6ms/step - loss: 2.2384 - accuracy: 0.3116 - val_loss: 1.9537 - val_accuracy: 0.4183
Epoch 4/200
1433/1433 [==============================] - 9s 6ms/step - loss: 1.9870 - accuracy: 0.3967 - val_loss: 1.7491 - val_accuracy: 0.4733
Epoch 5/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.7500 - accuracy: 0.4751 - val_loss: 1.4121 - val_accuracy: 0.5770
Epoch 6/200
1433/1433 [==============================] - 9s 6ms/step - loss: 1.5765 - accuracy: 0.5381 - val_loss: 1.4009 - val_accuracy: 0.5837
Epoch 7/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.4392 - accuracy: 0.5861 - val_loss: 1.1800 - val_accuracy: 0.6717
Epoch 8/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.3341 - accuracy: 0.6214 - val_loss: 1.0860 - val_accuracy: 0.7097
Epoch 9/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.2231 - accuracy: 0.6624 - val_loss: 0.9573 - val_accuracy: 0.7530
Epoch 10/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.1479 - accuracy: 0.6873 - val_loss: 0.9947 - val_accuracy: 0.7460
Epoch 11/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0792 - accuracy: 0.7164 - val_loss: 0.9694 - val_accuracy: 0.7457
Epoch 12/200
1433/1433 [==============================] - 8s 6ms/step - loss: 1.0103 - accuracy: 0.7361 - val_loss: 0.8014 - val_accuracy: 0.8090
Epoch 13/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.9437 - accuracy: 0.7582 - val_loss: 0.8463 - val_accuracy: 0.7837
Epoch 14/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.9066 - accuracy: 0.7718 - val_loss: 0.7541 - val_accuracy: 0.8263
Epoch 15/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.8615 - accuracy: 0.7895 - val_loss: 0.7027 - val_accuracy: 0.8393
Epoch 16/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.8126 - accuracy: 0.8071 - val_loss: 0.6592 - val_accuracy: 0.8553
Epoch 17/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.7702 - accuracy: 0.8183 - val_loss: 0.6497 - val_accuracy: 0.8547
Epoch 18/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.7419 - accuracy: 0.8255 - val_loss: 0.6245 - val_accuracy: 0.8597
Epoch 19/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.7130 - accuracy: 0.8346 - val_loss: 0.6994 - val_accuracy: 0.8360
Epoch 20/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.6844 - accuracy: 0.8475 - val_loss: 0.6092 - val_accuracy: 0.8627
Epoch 21/200
1433/1433 [==============================] - 9s 7ms/step - loss: 0.6512 - accuracy: 0.8537 - val_loss: 0.6424 - val_accuracy: 0.8613
Epoch 22/200
1433/1433 [==============================] - 9s 7ms/step - loss: 0.6196 - accuracy: 0.8662 - val_loss: 0.5572 - val_accuracy: 0.8873
Epoch 23/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.6120 - accuracy: 0.8700 - val_loss: 0.7342 - val_accuracy: 0.8337
Epoch 24/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.5908 - accuracy: 0.8737 - val_loss: 0.6160 - val_accuracy: 0.8683
Epoch 25/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.5662 - accuracy: 0.8846 - val_loss: 0.7433 - val_accuracy: 0.8343
Epoch 26/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.5550 - accuracy: 0.8874 - val_loss: 0.5193 - val_accuracy: 0.8967
Epoch 27/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.5382 - accuracy: 0.8922 - val_loss: 0.5782 - val_accuracy: 0.8847
Epoch 28/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.5158 - accuracy: 0.8993 - val_loss: 0.7551 - val_accuracy: 0.8297
Epoch 29/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.4985 - accuracy: 0.9044 - val_loss: 0.5391 - val_accuracy: 0.8907
Epoch 30/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.4807 - accuracy: 0.9118 - val_loss: 0.5662 - val_accuracy: 0.8903
Epoch 31/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4850 - accuracy: 0.9102 - val_loss: 0.4966 - val_accuracy: 0.9063
Epoch 32/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.4508 - accuracy: 0.9166 - val_loss: 0.5675 - val_accuracy: 0.8843
Epoch 33/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4422 - accuracy: 0.9199 - val_loss: 0.5242 - val_accuracy: 0.8953
Epoch 34/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.4293 - accuracy: 0.9255 - val_loss: 0.5623 - val_accuracy: 0.8853
Epoch 35/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4265 - accuracy: 0.9250 - val_loss: 0.5371 - val_accuracy: 0.8907
Epoch 36/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.4101 - accuracy: 0.9311 - val_loss: 0.5382 - val_accuracy: 0.8937
Epoch 37/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3927 - accuracy: 0.9346 - val_loss: 0.5356 - val_accuracy: 0.8950
Epoch 38/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3948 - accuracy: 0.9328 - val_loss: 0.5455 - val_accuracy: 0.8900
Epoch 39/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3782 - accuracy: 0.9390 - val_loss: 0.5090 - val_accuracy: 0.9000
Epoch 40/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3710 - accuracy: 0.9403 - val_loss: 0.7399 - val_accuracy: 0.8490
Epoch 41/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3578 - accuracy: 0.9416 - val_loss: 0.6294 - val_accuracy: 0.8710
Epoch 42/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3536 - accuracy: 0.9460 - val_loss: 0.4597 - val_accuracy: 0.9170
Epoch 43/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3554 - accuracy: 0.9444 - val_loss: 0.4906 - val_accuracy: 0.9043
Epoch 44/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3553 - accuracy: 0.9418 - val_loss: 0.4012 - val_accuracy: 0.9290
Epoch 45/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3391 - accuracy: 0.9479 - val_loss: 0.4534 - val_accuracy: 0.9207
Epoch 46/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3384 - accuracy: 0.9486 - val_loss: 0.4250 - val_accuracy: 0.9240
Epoch 47/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3242 - accuracy: 0.9499 - val_loss: 0.3975 - val_accuracy: 0.9350
Epoch 48/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.3241 - accuracy: 0.9495 - val_loss: 0.4207 - val_accuracy: 0.9263
Epoch 49/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3150 - accuracy: 0.9525 - val_loss: 0.4097 - val_accuracy: 0.9287
Epoch 50/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3083 - accuracy: 0.9553 - val_loss: 0.4595 - val_accuracy: 0.9123
Epoch 51/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.3016 - accuracy: 0.9566 - val_loss: 0.3855 - val_accuracy: 0.9357
Epoch 52/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.3000 - accuracy: 0.9560 - val_loss: 0.3686 - val_accuracy: 0.9420
Epoch 53/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2882 - accuracy: 0.9581 - val_loss: 0.4900 - val_accuracy: 0.9087
Epoch 54/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2926 - accuracy: 0.9555 - val_loss: 0.4095 - val_accuracy: 0.9313
Epoch 55/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2933 - accuracy: 0.9566 - val_loss: 0.3696 - val_accuracy: 0.9423
Epoch 56/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.2796 - accuracy: 0.9603 - val_loss: 0.3841 - val_accuracy: 0.9360
Epoch 57/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.2697 - accuracy: 0.9622 - val_loss: 0.4783 - val_accuracy: 0.9160
Epoch 58/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2712 - accuracy: 0.9637 - val_loss: 0.3972 - val_accuracy: 0.9367
Epoch 59/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2652 - accuracy: 0.9639 - val_loss: 0.7221 - val_accuracy: 0.8570
Epoch 60/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2636 - accuracy: 0.9621 - val_loss: 0.3596 - val_accuracy: 0.9427
Epoch 61/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2600 - accuracy: 0.9643 - val_loss: 0.4268 - val_accuracy: 0.9267
Epoch 62/200
1433/1433 [==============================] - 9s 7ms/step - loss: 0.2529 - accuracy: 0.9669 - val_loss: 0.4839 - val_accuracy: 0.9050
Epoch 63/200
1433/1433 [==============================] - 9s 7ms/step - loss: 0.2478 - accuracy: 0.9673 - val_loss: 0.3641 - val_accuracy: 0.9403
Epoch 64/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2453 - accuracy: 0.9682 - val_loss: 0.3978 - val_accuracy: 0.9250
Epoch 65/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2348 - accuracy: 0.9691 - val_loss: 0.5932 - val_accuracy: 0.8867
Epoch 66/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2470 - accuracy: 0.9664 - val_loss: 0.4858 - val_accuracy: 0.9080
Epoch 67/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2267 - accuracy: 0.9714 - val_loss: 0.4357 - val_accuracy: 0.9233
Epoch 68/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.2286 - accuracy: 0.9712 - val_loss: 0.3937 - val_accuracy: 0.9313
Epoch 69/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2286 - accuracy: 0.9693 - val_loss: 0.3838 - val_accuracy: 0.9337
Epoch 70/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2277 - accuracy: 0.9710 - val_loss: 0.3473 - val_accuracy: 0.9443
Epoch 71/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2211 - accuracy: 0.9708 - val_loss: 0.3680 - val_accuracy: 0.9363
Epoch 72/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2197 - accuracy: 0.9717 - val_loss: 0.5596 - val_accuracy: 0.8890
Epoch 73/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.2052 - accuracy: 0.9754 - val_loss: 0.6320 - val_accuracy: 0.8743
Epoch 74/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.2189 - accuracy: 0.9704 - val_loss: 0.4124 - val_accuracy: 0.9277
Epoch 75/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2137 - accuracy: 0.9724 - val_loss: 0.3889 - val_accuracy: 0.9323
Epoch 76/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.2126 - accuracy: 0.9726 - val_loss: 0.4447 - val_accuracy: 0.9150
Epoch 77/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.2061 - accuracy: 0.9740 - val_loss: 0.4534 - val_accuracy: 0.9187
Epoch 78/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.2071 - accuracy: 0.9746 - val_loss: 0.4018 - val_accuracy: 0.9247
Epoch 79/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1962 - accuracy: 0.9763 - val_loss: 0.3260 - val_accuracy: 0.9437
Epoch 80/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1946 - accuracy: 0.9763 - val_loss: 0.3318 - val_accuracy: 0.9483
Epoch 81/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1935 - accuracy: 0.9776 - val_loss: 0.3536 - val_accuracy: 0.9370
Epoch 82/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1936 - accuracy: 0.9765 - val_loss: 0.3807 - val_accuracy: 0.9333
Epoch 83/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1863 - accuracy: 0.9777 - val_loss: 0.3380 - val_accuracy: 0.9447
Epoch 84/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1846 - accuracy: 0.9776 - val_loss: 0.4556 - val_accuracy: 0.9107
Epoch 85/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1834 - accuracy: 0.9779 - val_loss: 0.4041 - val_accuracy: 0.9280
Epoch 86/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1814 - accuracy: 0.9790 - val_loss: 0.3558 - val_accuracy: 0.9420
Epoch 87/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1887 - accuracy: 0.9756 - val_loss: 0.3910 - val_accuracy: 0.9313
Epoch 88/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1772 - accuracy: 0.9800 - val_loss: 0.3253 - val_accuracy: 0.9500
Epoch 89/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1728 - accuracy: 0.9807 - val_loss: 0.2959 - val_accuracy: 0.9527
Epoch 90/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1802 - accuracy: 0.9780 - val_loss: 0.3205 - val_accuracy: 0.9457
Epoch 91/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1712 - accuracy: 0.9807 - val_loss: 0.3510 - val_accuracy: 0.9340
Epoch 92/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1690 - accuracy: 0.9813 - val_loss: 0.3453 - val_accuracy: 0.9400
Epoch 93/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1687 - accuracy: 0.9798 - val_loss: 0.3068 - val_accuracy: 0.9493
Epoch 94/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1698 - accuracy: 0.9802 - val_loss: 0.3753 - val_accuracy: 0.9290
Epoch 95/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1716 - accuracy: 0.9786 - val_loss: 0.3339 - val_accuracy: 0.9430
Epoch 96/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1664 - accuracy: 0.9798 - val_loss: 0.3163 - val_accuracy: 0.9503
Epoch 97/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1635 - accuracy: 0.9803 - val_loss: 0.3166 - val_accuracy: 0.9473
Epoch 98/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1577 - accuracy: 0.9830 - val_loss: 0.3006 - val_accuracy: 0.9500
Epoch 99/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1594 - accuracy: 0.9829 - val_loss: 0.3057 - val_accuracy: 0.9497
Epoch 100/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1598 - accuracy: 0.9814 - val_loss: 0.4492 - val_accuracy: 0.9150
Epoch 101/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1605 - accuracy: 0.9793 - val_loss: 0.3098 - val_accuracy: 0.9457
Epoch 102/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1501 - accuracy: 0.9844 - val_loss: 0.3090 - val_accuracy: 0.9490
Epoch 103/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1597 - accuracy: 0.9793 - val_loss: 0.3321 - val_accuracy: 0.9403
Epoch 104/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1571 - accuracy: 0.9806 - val_loss: 0.4436 - val_accuracy: 0.9213
Epoch 105/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1536 - accuracy: 0.9820 - val_loss: 0.4681 - val_accuracy: 0.9153
Epoch 106/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1476 - accuracy: 0.9833 - val_loss: 0.4051 - val_accuracy: 0.9243
Epoch 107/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1471 - accuracy: 0.9825 - val_loss: 0.3132 - val_accuracy: 0.9453
Epoch 108/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1442 - accuracy: 0.9830 - val_loss: 0.2949 - val_accuracy: 0.9543
Epoch 109/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1434 - accuracy: 0.9835 - val_loss: 0.3683 - val_accuracy: 0.9340
Epoch 110/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1394 - accuracy: 0.9852 - val_loss: 0.3996 - val_accuracy: 0.9287
Epoch 111/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1416 - accuracy: 0.9842 - val_loss: 0.4042 - val_accuracy: 0.9287
Epoch 112/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1435 - accuracy: 0.9839 - val_loss: 0.3243 - val_accuracy: 0.9450
Epoch 113/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1413 - accuracy: 0.9844 - val_loss: 0.3839 - val_accuracy: 0.9310
Epoch 114/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1355 - accuracy: 0.9846 - val_loss: 0.3324 - val_accuracy: 0.9440
Epoch 115/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1389 - accuracy: 0.9847 - val_loss: 0.3199 - val_accuracy: 0.9423
Epoch 116/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1428 - accuracy: 0.9841 - val_loss: 0.3311 - val_accuracy: 0.9407
Epoch 117/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1291 - accuracy: 0.9866 - val_loss: 0.4212 - val_accuracy: 0.9223
Epoch 118/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1369 - accuracy: 0.9850 - val_loss: 0.3215 - val_accuracy: 0.9460
Epoch 119/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1360 - accuracy: 0.9847 - val_loss: 0.3727 - val_accuracy: 0.9277
Epoch 120/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1300 - accuracy: 0.9864 - val_loss: 0.3197 - val_accuracy: 0.9460
Epoch 121/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1318 - accuracy: 0.9855 - val_loss: 0.2931 - val_accuracy: 0.9477
Epoch 122/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1224 - accuracy: 0.9867 - val_loss: 0.2885 - val_accuracy: 0.9540
Epoch 123/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1350 - accuracy: 0.9842 - val_loss: 0.3339 - val_accuracy: 0.9440
Epoch 124/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1307 - accuracy: 0.9844 - val_loss: 0.4463 - val_accuracy: 0.9250
Epoch 125/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1234 - accuracy: 0.9867 - val_loss: 0.3152 - val_accuracy: 0.9457
Epoch 126/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1250 - accuracy: 0.9860 - val_loss: 0.3058 - val_accuracy: 0.9460
Epoch 127/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1249 - accuracy: 0.9862 - val_loss: 0.3603 - val_accuracy: 0.9343
Epoch 128/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1233 - accuracy: 0.9864 - val_loss: 0.2889 - val_accuracy: 0.9500
Epoch 129/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1199 - accuracy: 0.9869 - val_loss: 0.3523 - val_accuracy: 0.9337
Epoch 130/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1254 - accuracy: 0.9864 - val_loss: 0.2843 - val_accuracy: 0.9487
Epoch 131/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1234 - accuracy: 0.9865 - val_loss: 0.2909 - val_accuracy: 0.9463
Epoch 132/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1218 - accuracy: 0.9865 - val_loss: 0.4997 - val_accuracy: 0.9017
Epoch 133/200
1433/1433 [==============================] - 8s 6ms/step - loss: 0.1120 - accuracy: 0.9890 - val_loss: 0.3413 - val_accuracy: 0.9410
Epoch 134/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1169 - accuracy: 0.9867 - val_loss: 0.3236 - val_accuracy: 0.9443
Epoch 135/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1195 - accuracy: 0.9870 - val_loss: 0.3236 - val_accuracy: 0.9420
Epoch 136/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1166 - accuracy: 0.9876 - val_loss: 0.3571 - val_accuracy: 0.9313
Epoch 137/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1106 - accuracy: 0.9899 - val_loss: 0.2814 - val_accuracy: 0.9527
Epoch 138/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1158 - accuracy: 0.9873 - val_loss: 0.3017 - val_accuracy: 0.9517
Epoch 139/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1225 - accuracy: 0.9862 - val_loss: 0.3083 - val_accuracy: 0.9490
Epoch 140/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1121 - accuracy: 0.9882 - val_loss: 0.2927 - val_accuracy: 0.9503
Epoch 141/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1176 - accuracy: 0.9867 - val_loss: 0.3177 - val_accuracy: 0.9427
Epoch 142/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1122 - accuracy: 0.9874 - val_loss: 0.4356 - val_accuracy: 0.9210
Epoch 143/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1075 - accuracy: 0.9892 - val_loss: 0.3123 - val_accuracy: 0.9460
Epoch 144/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1095 - accuracy: 0.9883 - val_loss: 0.2607 - val_accuracy: 0.9553
Epoch 145/200
1433/1433 [==============================] - 9s 7ms/step - loss: 0.1071 - accuracy: 0.9890 - val_loss: 0.3211 - val_accuracy: 0.9420
Epoch 146/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1091 - accuracy: 0.9884 - val_loss: 0.3211 - val_accuracy: 0.9363
Epoch 147/200
1433/1433 [==============================] - 9s 7ms/step - loss: 0.1156 - accuracy: 0.9869 - val_loss: 0.3201 - val_accuracy: 0.9440
Epoch 148/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1017 - accuracy: 0.9897 - val_loss: 0.3789 - val_accuracy: 0.9313
Epoch 149/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1072 - accuracy: 0.9877 - val_loss: 0.6108 - val_accuracy: 0.8877
Epoch 150/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1034 - accuracy: 0.9893 - val_loss: 0.3205 - val_accuracy: 0.9400
Epoch 151/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1021 - accuracy: 0.9887 - val_loss: 0.3895 - val_accuracy: 0.9337
Epoch 152/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1032 - accuracy: 0.9884 - val_loss: 0.3191 - val_accuracy: 0.9473
Epoch 153/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1024 - accuracy: 0.9899 - val_loss: 0.3384 - val_accuracy: 0.9350
Epoch 154/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1058 - accuracy: 0.9880 - val_loss: 0.3029 - val_accuracy: 0.9480
Epoch 155/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1005 - accuracy: 0.9895 - val_loss: 0.2628 - val_accuracy: 0.9523
Epoch 156/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0994 - accuracy: 0.9894 - val_loss: 0.5165 - val_accuracy: 0.8937
Epoch 157/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1035 - accuracy: 0.9881 - val_loss: 0.4136 - val_accuracy: 0.9223
Epoch 158/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0951 - accuracy: 0.9895 - val_loss: 0.3645 - val_accuracy: 0.9320
Epoch 159/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0961 - accuracy: 0.9903 - val_loss: 0.3404 - val_accuracy: 0.9367
Epoch 160/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0991 - accuracy: 0.9892 - val_loss: 0.2983 - val_accuracy: 0.9507
Epoch 161/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0945 - accuracy: 0.9908 - val_loss: 0.3014 - val_accuracy: 0.9513
Epoch 162/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0977 - accuracy: 0.9902 - val_loss: 0.2682 - val_accuracy: 0.9540
Epoch 163/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1000 - accuracy: 0.9891 - val_loss: 0.2614 - val_accuracy: 0.9580
Epoch 164/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0958 - accuracy: 0.9894 - val_loss: 0.4779 - val_accuracy: 0.9107
Epoch 165/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0923 - accuracy: 0.9902 - val_loss: 0.7218 - val_accuracy: 0.8800
Epoch 166/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.1001 - accuracy: 0.9892 - val_loss: 0.2686 - val_accuracy: 0.9560
Epoch 167/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0986 - accuracy: 0.9887 - val_loss: 0.3769 - val_accuracy: 0.9330
Epoch 168/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0935 - accuracy: 0.9909 - val_loss: 0.3438 - val_accuracy: 0.9423
Epoch 169/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0924 - accuracy: 0.9904 - val_loss: 0.3111 - val_accuracy: 0.9413
Epoch 170/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0917 - accuracy: 0.9902 - val_loss: 0.2536 - val_accuracy: 0.9560
Epoch 171/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0931 - accuracy: 0.9901 - val_loss: 0.2981 - val_accuracy: 0.9550
Epoch 172/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0932 - accuracy: 0.9898 - val_loss: 0.2785 - val_accuracy: 0.9543
Epoch 173/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0863 - accuracy: 0.9920 - val_loss: 0.3523 - val_accuracy: 0.9380
Epoch 174/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0922 - accuracy: 0.9895 - val_loss: 0.3388 - val_accuracy: 0.9423
Epoch 175/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0904 - accuracy: 0.9898 - val_loss: 0.3531 - val_accuracy: 0.9390
Epoch 176/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0935 - accuracy: 0.9900 - val_loss: 0.2764 - val_accuracy: 0.9550
Epoch 177/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0920 - accuracy: 0.9898 - val_loss: 0.3275 - val_accuracy: 0.9427
Epoch 178/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0913 - accuracy: 0.9902 - val_loss: 0.2956 - val_accuracy: 0.9527
Epoch 179/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0892 - accuracy: 0.9913 - val_loss: 0.2605 - val_accuracy: 0.9587
Epoch 180/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0863 - accuracy: 0.9918 - val_loss: 0.2603 - val_accuracy: 0.9583
Epoch 181/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0868 - accuracy: 0.9919 - val_loss: 0.2864 - val_accuracy: 0.9490
Epoch 182/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0865 - accuracy: 0.9908 - val_loss: 0.2818 - val_accuracy: 0.9503
Epoch 183/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0827 - accuracy: 0.9915 - val_loss: 0.3078 - val_accuracy: 0.9457
Epoch 184/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0919 - accuracy: 0.9892 - val_loss: 0.2854 - val_accuracy: 0.9553
Epoch 185/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0857 - accuracy: 0.9916 - val_loss: 0.3467 - val_accuracy: 0.9367
Epoch 186/200
1433/1433 [==============================] - 10s 7ms/step - loss: 0.0889 - accuracy: 0.9900 - val_loss: 0.5024 - val_accuracy: 0.9090
Epoch 187/200
1433/1433 [==============================] - 9s 7ms/step - loss: 0.0875 - accuracy: 0.9906 - val_loss: 0.4934 - val_accuracy: 0.9060
Epoch 188/200
1433/1433 [==============================] - 9s 7ms/step - loss: 0.0911 - accuracy: 0.9902 - val_loss: 0.3478 - val_accuracy: 0.9373
Epoch 189/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0807 - accuracy: 0.9916 - val_loss: 0.3230 - val_accuracy: 0.9430
Epoch 190/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0898 - accuracy: 0.9899 - val_loss: 0.2685 - val_accuracy: 0.9570
Epoch 191/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0793 - accuracy: 0.9929 - val_loss: 0.2624 - val_accuracy: 0.9537
Epoch 192/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0803 - accuracy: 0.9921 - val_loss: 0.2815 - val_accuracy: 0.9500
Epoch 193/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0835 - accuracy: 0.9918 - val_loss: 0.2812 - val_accuracy: 0.9527
Epoch 194/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0803 - accuracy: 0.9907 - val_loss: 0.3137 - val_accuracy: 0.9460
Epoch 195/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0787 - accuracy: 0.9913 - val_loss: 0.3555 - val_accuracy: 0.9360
Epoch 196/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0759 - accuracy: 0.9925 - val_loss: 0.2603 - val_accuracy: 0.9580
Epoch 197/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0819 - accuracy: 0.9916 - val_loss: 0.3024 - val_accuracy: 0.9473
Epoch 198/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0767 - accuracy: 0.9925 - val_loss: 0.3172 - val_accuracy: 0.9463
Epoch 199/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0849 - accuracy: 0.9906 - val_loss: 0.3185 - val_accuracy: 0.9453
Epoch 200/200
1433/1433 [==============================] - 9s 6ms/step - loss: 0.0792 - accuracy: 0.9918 - val_loss: 0.2780 - val_accuracy: 0.9513

Conv2D_128_improved_aug.summary()

Model: "Conv2D_128_Final_Augmentation"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalised_data (Sequential  (None, None, None, 1)    0         
 )                                                               
                                                                 
 conv2d (Conv2D)             (None, 32, 32, 64)        1664      
                                                                 
 max_pooling2d (MaxPooling2D  (None, 16, 16, 64)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 8, 8, 128)        0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 4, 4, 128)        0         
 2D)                                                             
                                                                 
 conv2d_3 (Conv2D)           (None, 4, 4, 256)         295168    
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 2, 2, 256)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 1024)              0         
                                                                 
 dropout (Dropout)           (None, 1024)              0         
                                                                 
 dense (Dense)               (None, 1024)              1049600   
                                                                 
 dropout_1 (Dropout)         (None, 1024)              0         
                                                                 
 dense_1 (Dense)             (None, 128)               131200    
                                                                 
 dropout_2 (Dropout)         (None, 128)               0         
                                                                 
 dense_2 (Dense)             (None, 15)                1935      
                                                                 
=================================================================
Total params: 1,701,007
Trainable params: 1,701,007
Non-trainable params: 0
_________________________________________________________________

plot_learning_curve(Conv2D_128_improved_aug_history.history)

Observations

• Validation accuracy and loss curve are converging towards the training curve, which shows the decrease in learning rate and regularisation do indeed improve the accuracy.

Conv2D_128_improved_aug.evaluate(test_data_128.batch(10))

300/300 [==============================] - 2s 7ms/step - loss: 0.2710 - accuracy: 0.9493

[0.2709842622280121, 0.9493333101272583]

Observations

• Testing accuracy is quite high for this curve, accuracy of 90% is achieved.

31 x 31 images

Time-based Decay

# Try for 31 x 31 images
tf.keras.backend.clear_session()

Conv2D_31_lr = Sequential(name="Conv2D_31V1",
    layers = [
        normalised_data,
        Conv2D(32, (3, 3),input_shape=(31, 31 ,1), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),
        Conv2D(64, (3, 3), activation='relu', padding='same'),
        Conv2D(128, (3, 3), activation='relu', padding='same'),
        Conv2D(128, (3, 3), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.5),

        Dense(512, activation='relu'),
        Dropout(0.5),
        Dense(256, activation='relu'),
        Dropout(0.5),
        Dense(256, activation='relu'),
        Dropout(0.5),

        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.00025)
Conv2D_31_lr.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_31_lr.build(input_shape=(None, 31, 31, 1))

Conv2D_31_lr_history = Conv2D_31_lr.fit(
    train_data_31.batch(10), 
    epochs=200,
    validation_data=val_data_31.batch(10),
    callbacks=[lr_schedule]
    )

Epoch 1: LearningRateScheduler setting learning rate to 0.00025000001031186275.
Epoch 1/200
903/903 [==============================] - 8s 8ms/step - loss: 2.5878 - accuracy: 0.1120 - val_loss: 2.4884 - val_accuracy: 0.1540 - lr: 2.5000e-04

Epoch 2: LearningRateScheduler setting learning rate to 0.00025000001109311276.
Epoch 2/200
903/903 [==============================] - 7s 8ms/step - loss: 2.2192 - accuracy: 0.2559 - val_loss: 2.0248 - val_accuracy: 0.3647 - lr: 2.5000e-04

Epoch 3: LearningRateScheduler setting learning rate to 0.0002500000113535295.
Epoch 3/200
903/903 [==============================] - 7s 8ms/step - loss: 1.8557 - accuracy: 0.4021 - val_loss: 1.5971 - val_accuracy: 0.4763 - lr: 2.5000e-04

Epoch 4: LearningRateScheduler setting learning rate to 0.00025000001148373785.
Epoch 4/200
903/903 [==============================] - 7s 8ms/step - loss: 1.5413 - accuracy: 0.5010 - val_loss: 1.3127 - val_accuracy: 0.5763 - lr: 2.5000e-04

Epoch 5: LearningRateScheduler setting learning rate to 0.0002500000115618628.
Epoch 5/200
903/903 [==============================] - 7s 8ms/step - loss: 1.2933 - accuracy: 0.5918 - val_loss: 1.0985 - val_accuracy: 0.6490 - lr: 2.5000e-04

Epoch 6: LearningRateScheduler setting learning rate to 0.00025000001161394614.
Epoch 6/200
903/903 [==============================] - 7s 8ms/step - loss: 1.1053 - accuracy: 0.6504 - val_loss: 0.9451 - val_accuracy: 0.6870 - lr: 2.5000e-04

Epoch 7: LearningRateScheduler setting learning rate to 0.00025000001165114853.
Epoch 7/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9145 - accuracy: 0.7154 - val_loss: 0.7944 - val_accuracy: 0.7503 - lr: 2.5000e-04

Epoch 8: LearningRateScheduler setting learning rate to 0.0002500000116790503.
Epoch 8/200
903/903 [==============================] - 7s 8ms/step - loss: 0.8031 - accuracy: 0.7472 - val_loss: 0.6584 - val_accuracy: 0.7980 - lr: 2.5000e-04

Epoch 9: LearningRateScheduler setting learning rate to 0.0002500000117007517.
Epoch 9/200
903/903 [==============================] - 7s 8ms/step - loss: 0.7046 - accuracy: 0.7804 - val_loss: 0.5697 - val_accuracy: 0.8253 - lr: 2.5000e-04

Epoch 10: LearningRateScheduler setting learning rate to 0.0002500000117181128.
Epoch 10/200
903/903 [==============================] - 7s 8ms/step - loss: 0.6014 - accuracy: 0.8156 - val_loss: 0.5595 - val_accuracy: 0.8347 - lr: 2.5000e-04

Epoch 11: LearningRateScheduler setting learning rate to 0.0002500000117323174.
Epoch 11/200
903/903 [==============================] - 7s 8ms/step - loss: 0.5421 - accuracy: 0.8307 - val_loss: 0.5073 - val_accuracy: 0.8463 - lr: 2.5000e-04

Epoch 12: LearningRateScheduler setting learning rate to 0.0002500000117441545.
Epoch 12/200
903/903 [==============================] - 7s 8ms/step - loss: 0.4709 - accuracy: 0.8576 - val_loss: 0.4302 - val_accuracy: 0.8673 - lr: 2.5000e-04

Epoch 13: LearningRateScheduler setting learning rate to 0.0002500000117541705.
Epoch 13/200
903/903 [==============================] - 8s 8ms/step - loss: 0.4247 - accuracy: 0.8672 - val_loss: 0.4482 - val_accuracy: 0.8640 - lr: 2.5000e-04

Epoch 14: LearningRateScheduler setting learning rate to 0.00025000001176275565.
Epoch 14/200
903/903 [==============================] - 7s 8ms/step - loss: 0.3884 - accuracy: 0.8795 - val_loss: 0.3427 - val_accuracy: 0.8987 - lr: 2.5000e-04

Epoch 15: LearningRateScheduler setting learning rate to 0.00025000001177019615.
Epoch 15/200
903/903 [==============================] - 7s 8ms/step - loss: 0.3482 - accuracy: 0.8932 - val_loss: 0.4077 - val_accuracy: 0.8783 - lr: 2.5000e-04

Epoch 16: LearningRateScheduler setting learning rate to 0.00025000001177670657.
Epoch 16/200
903/903 [==============================] - 7s 8ms/step - loss: 0.3117 - accuracy: 0.9058 - val_loss: 0.5169 - val_accuracy: 0.8580 - lr: 2.5000e-04

Epoch 17: LearningRateScheduler setting learning rate to 0.00025000001178245105.
Epoch 17/200
903/903 [==============================] - 7s 8ms/step - loss: 0.2971 - accuracy: 0.9095 - val_loss: 0.3723 - val_accuracy: 0.8927 - lr: 2.5000e-04

Epoch 18: LearningRateScheduler setting learning rate to 0.00025000001178755726.
Epoch 18/200
903/903 [==============================] - 7s 8ms/step - loss: 0.2859 - accuracy: 0.9144 - val_loss: 0.3144 - val_accuracy: 0.9083 - lr: 2.5000e-04

Epoch 19: LearningRateScheduler setting learning rate to 0.000250000011792126.
Epoch 19/200
903/903 [==============================] - 7s 8ms/step - loss: 0.2435 - accuracy: 0.9249 - val_loss: 0.3547 - val_accuracy: 0.9027 - lr: 2.5000e-04

Epoch 20: LearningRateScheduler setting learning rate to 0.0002500000117962378.
Epoch 20/200
903/903 [==============================] - 7s 8ms/step - loss: 0.2310 - accuracy: 0.9294 - val_loss: 0.3312 - val_accuracy: 0.9053 - lr: 2.5000e-04

Epoch 21: LearningRateScheduler setting learning rate to 0.00025000001179995804.
Epoch 21/200
903/903 [==============================] - 7s 8ms/step - loss: 0.2140 - accuracy: 0.9346 - val_loss: 0.3525 - val_accuracy: 0.9030 - lr: 2.5000e-04

Epoch 22: LearningRateScheduler setting learning rate to 0.0002500000118033401.
Epoch 22/200
903/903 [==============================] - 7s 8ms/step - loss: 0.2045 - accuracy: 0.9354 - val_loss: 0.3437 - val_accuracy: 0.9070 - lr: 2.5000e-04

Epoch 23: LearningRateScheduler setting learning rate to 0.0002500000118064281.
Epoch 23/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1983 - accuracy: 0.9434 - val_loss: 0.3561 - val_accuracy: 0.9073 - lr: 2.5000e-04

Epoch 24: LearningRateScheduler setting learning rate to 0.00025000001180925865.
Epoch 24/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1903 - accuracy: 0.9434 - val_loss: 0.2916 - val_accuracy: 0.9223 - lr: 2.5000e-04

Epoch 25: LearningRateScheduler setting learning rate to 0.0002500000118118628.
Epoch 25/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1675 - accuracy: 0.9484 - val_loss: 0.4124 - val_accuracy: 0.9033 - lr: 2.5000e-04

Epoch 26: LearningRateScheduler setting learning rate to 0.0002500000118142667.
Epoch 26/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1809 - accuracy: 0.9485 - val_loss: 0.3673 - val_accuracy: 0.9027 - lr: 2.5000e-04

Epoch 27: LearningRateScheduler setting learning rate to 0.00025000001181649245.
Epoch 27/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1592 - accuracy: 0.9526 - val_loss: 0.5301 - val_accuracy: 0.8790 - lr: 2.5000e-04

Epoch 28: LearningRateScheduler setting learning rate to 0.00025000001181855927.
Epoch 28/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1626 - accuracy: 0.9519 - val_loss: 0.2825 - val_accuracy: 0.9243 - lr: 2.5000e-04

Epoch 29: LearningRateScheduler setting learning rate to 0.0002500000118204835.
Epoch 29/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1418 - accuracy: 0.9586 - val_loss: 0.2775 - val_accuracy: 0.9250 - lr: 2.5000e-04

Epoch 30: LearningRateScheduler setting learning rate to 0.0002500000118222795.
Epoch 30/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1499 - accuracy: 0.9541 - val_loss: 0.3070 - val_accuracy: 0.9220 - lr: 2.5000e-04

Epoch 31: LearningRateScheduler setting learning rate to 0.00025000001182395957.
Epoch 31/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1350 - accuracy: 0.9613 - val_loss: 0.2828 - val_accuracy: 0.9280 - lr: 2.5000e-04

Epoch 32: LearningRateScheduler setting learning rate to 0.0002500000118255347.
Epoch 32/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1176 - accuracy: 0.9646 - val_loss: 0.6029 - val_accuracy: 0.8603 - lr: 2.5000e-04

Epoch 33: LearningRateScheduler setting learning rate to 0.00025000001182701436.
Epoch 33/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1219 - accuracy: 0.9643 - val_loss: 0.2967 - val_accuracy: 0.9300 - lr: 2.5000e-04

Epoch 34: LearningRateScheduler setting learning rate to 0.0002500000118284069.
Epoch 34/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1274 - accuracy: 0.9612 - val_loss: 0.2797 - val_accuracy: 0.9227 - lr: 2.5000e-04

Epoch 35: LearningRateScheduler setting learning rate to 0.00025000001182972.
Epoch 35/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1112 - accuracy: 0.9689 - val_loss: 0.2537 - val_accuracy: 0.9360 - lr: 2.5000e-04

Epoch 36: LearningRateScheduler setting learning rate to 0.00025000001183096004.
Epoch 36/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1121 - accuracy: 0.9680 - val_loss: 0.3302 - val_accuracy: 0.9230 - lr: 2.5000e-04

Epoch 37: LearningRateScheduler setting learning rate to 0.0002500000118321331.
Epoch 37/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1012 - accuracy: 0.9684 - val_loss: 0.3028 - val_accuracy: 0.9313 - lr: 2.5000e-04

Epoch 38: LearningRateScheduler setting learning rate to 0.0002500000118332444.
Epoch 38/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1055 - accuracy: 0.9690 - val_loss: 0.3312 - val_accuracy: 0.9260 - lr: 2.5000e-04

Epoch 39: LearningRateScheduler setting learning rate to 0.0002500000118342987.
Epoch 39/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1114 - accuracy: 0.9678 - val_loss: 0.2876 - val_accuracy: 0.9277 - lr: 2.5000e-04

Epoch 40: LearningRateScheduler setting learning rate to 0.0002500000118353003.
Epoch 40/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1070 - accuracy: 0.9680 - val_loss: 0.4494 - val_accuracy: 0.8917 - lr: 2.5000e-04

Epoch 41: LearningRateScheduler setting learning rate to 0.00025000001183625307.
Epoch 41/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1085 - accuracy: 0.9683 - val_loss: 0.3155 - val_accuracy: 0.9243 - lr: 2.5000e-04

Epoch 42: LearningRateScheduler setting learning rate to 0.00025000001183716043.
Epoch 42/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0847 - accuracy: 0.9751 - val_loss: 0.2639 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 43: LearningRateScheduler setting learning rate to 0.00025000001183802563.
Epoch 43/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0957 - accuracy: 0.9711 - val_loss: 0.3405 - val_accuracy: 0.9247 - lr: 2.5000e-04

Epoch 44: LearningRateScheduler setting learning rate to 0.00025000001183885146.
Epoch 44/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0956 - accuracy: 0.9719 - val_loss: 0.3981 - val_accuracy: 0.9077 - lr: 2.5000e-04

Epoch 45: LearningRateScheduler setting learning rate to 0.0002500000118396406.
Epoch 45/200
903/903 [==============================] - 7s 8ms/step - loss: 0.1051 - accuracy: 0.9704 - val_loss: 0.3518 - val_accuracy: 0.9190 - lr: 2.5000e-04

Epoch 46: LearningRateScheduler setting learning rate to 0.0002500000118403954.
Epoch 46/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0907 - accuracy: 0.9742 - val_loss: 0.2666 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 47: LearningRateScheduler setting learning rate to 0.00025000001184111815.
Epoch 47/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0820 - accuracy: 0.9773 - val_loss: 0.3318 - val_accuracy: 0.9243 - lr: 2.5000e-04

Epoch 48: LearningRateScheduler setting learning rate to 0.00025000001184181074.
Epoch 48/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0965 - accuracy: 0.9730 - val_loss: 0.2792 - val_accuracy: 0.9297 - lr: 2.5000e-04

Epoch 49: LearningRateScheduler setting learning rate to 0.00025000001184247503.
Epoch 49/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0819 - accuracy: 0.9756 - val_loss: 0.3053 - val_accuracy: 0.9287 - lr: 2.5000e-04

Epoch 50: LearningRateScheduler setting learning rate to 0.0002500000118431128.
Epoch 50/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0743 - accuracy: 0.9792 - val_loss: 0.2941 - val_accuracy: 0.9320 - lr: 2.5000e-04

Epoch 51: LearningRateScheduler setting learning rate to 0.00025000001184372555.
Epoch 51/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0892 - accuracy: 0.9771 - val_loss: 0.2931 - val_accuracy: 0.9340 - lr: 2.5000e-04

Epoch 52: LearningRateScheduler setting learning rate to 0.00025000001184431476.
Epoch 52/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0907 - accuracy: 0.9757 - val_loss: 0.2704 - val_accuracy: 0.9353 - lr: 2.5000e-04

Epoch 53: LearningRateScheduler setting learning rate to 0.0002500000118448817.
Epoch 53/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0864 - accuracy: 0.9753 - val_loss: 0.2782 - val_accuracy: 0.9333 - lr: 2.5000e-04

Epoch 54: LearningRateScheduler setting learning rate to 0.00025000001184542764.
Epoch 54/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0771 - accuracy: 0.9777 - val_loss: 0.3016 - val_accuracy: 0.9313 - lr: 2.5000e-04

Epoch 55: LearningRateScheduler setting learning rate to 0.0002500000118459537.
Epoch 55/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0719 - accuracy: 0.9787 - val_loss: 0.2763 - val_accuracy: 0.9403 - lr: 2.5000e-04

Epoch 56: LearningRateScheduler setting learning rate to 0.00025000001184646105.
Epoch 56/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0807 - accuracy: 0.9775 - val_loss: 0.3040 - val_accuracy: 0.9327 - lr: 2.5000e-04

Epoch 57: LearningRateScheduler setting learning rate to 0.0002500000118469505.
Epoch 57/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0793 - accuracy: 0.9783 - val_loss: 0.3079 - val_accuracy: 0.9330 - lr: 2.5000e-04

Epoch 58: LearningRateScheduler setting learning rate to 0.00025000001184742317.
Epoch 58/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0815 - accuracy: 0.9790 - val_loss: 0.3837 - val_accuracy: 0.9140 - lr: 2.5000e-04

Epoch 59: LearningRateScheduler setting learning rate to 0.0002500000118478798.
Epoch 59/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0762 - accuracy: 0.9787 - val_loss: 0.3066 - val_accuracy: 0.9323 - lr: 2.5000e-04

Epoch 60: LearningRateScheduler setting learning rate to 0.00025000001184832116.
Epoch 60/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0602 - accuracy: 0.9814 - val_loss: 0.3136 - val_accuracy: 0.9297 - lr: 2.5000e-04

Epoch 61: LearningRateScheduler setting learning rate to 0.0002500000118487481.
Epoch 61/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0679 - accuracy: 0.9800 - val_loss: 0.3160 - val_accuracy: 0.9370 - lr: 2.5000e-04

Epoch 62: LearningRateScheduler setting learning rate to 0.0002500000118491612.
Epoch 62/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0753 - accuracy: 0.9804 - val_loss: 0.2959 - val_accuracy: 0.9313 - lr: 2.5000e-04

Epoch 63: LearningRateScheduler setting learning rate to 0.0002500000118495612.
Epoch 63/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0829 - accuracy: 0.9764 - val_loss: 0.2854 - val_accuracy: 0.9330 - lr: 2.5000e-04

Epoch 64: LearningRateScheduler setting learning rate to 0.00025000001184994876.
Epoch 64/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0511 - accuracy: 0.9873 - val_loss: 0.2710 - val_accuracy: 0.9397 - lr: 2.5000e-04

Epoch 65: LearningRateScheduler setting learning rate to 0.0002500000118503244.
Epoch 65/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0813 - accuracy: 0.9773 - val_loss: 0.3056 - val_accuracy: 0.9313 - lr: 2.5000e-04

Epoch 66: LearningRateScheduler setting learning rate to 0.0002500000118506886.
Epoch 66/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0823 - accuracy: 0.9777 - val_loss: 0.2922 - val_accuracy: 0.9397 - lr: 2.5000e-04

Epoch 67: LearningRateScheduler setting learning rate to 0.0002500000118510419.
Epoch 67/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0542 - accuracy: 0.9840 - val_loss: 0.2706 - val_accuracy: 0.9427 - lr: 2.5000e-04

Epoch 68: LearningRateScheduler setting learning rate to 0.0002500000118513849.
Epoch 68/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0632 - accuracy: 0.9829 - val_loss: 0.3098 - val_accuracy: 0.9313 - lr: 2.5000e-04

Epoch 69: LearningRateScheduler setting learning rate to 0.0002500000118517179.
Epoch 69/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0669 - accuracy: 0.9804 - val_loss: 0.2905 - val_accuracy: 0.9337 - lr: 2.5000e-04

Epoch 70: LearningRateScheduler setting learning rate to 0.0002500000118520414.
Epoch 70/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0621 - accuracy: 0.9834 - val_loss: 0.2791 - val_accuracy: 0.9357 - lr: 2.5000e-04

Epoch 71: LearningRateScheduler setting learning rate to 0.00025000001185235574.
Epoch 71/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0752 - accuracy: 0.9791 - val_loss: 0.2630 - val_accuracy: 0.9397 - lr: 2.5000e-04

Epoch 72: LearningRateScheduler setting learning rate to 0.00025000001185266144.
Epoch 72/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0617 - accuracy: 0.9836 - val_loss: 0.2773 - val_accuracy: 0.9390 - lr: 2.5000e-04

Epoch 73: LearningRateScheduler setting learning rate to 0.0002500000118529587.
Epoch 73/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0709 - accuracy: 0.9808 - val_loss: 0.3980 - val_accuracy: 0.9150 - lr: 2.5000e-04

Epoch 74: LearningRateScheduler setting learning rate to 0.00025000001185324793.
Epoch 74/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0598 - accuracy: 0.9824 - val_loss: 0.3091 - val_accuracy: 0.9327 - lr: 2.5000e-04

Epoch 75: LearningRateScheduler setting learning rate to 0.0002500000118535295.
Epoch 75/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0662 - accuracy: 0.9821 - val_loss: 0.2884 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 76: LearningRateScheduler setting learning rate to 0.0002500000118538036.
Epoch 76/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0728 - accuracy: 0.9825 - val_loss: 0.3696 - val_accuracy: 0.9203 - lr: 2.5000e-04

Epoch 77: LearningRateScheduler setting learning rate to 0.00025000001185407063.
Epoch 77/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0469 - accuracy: 0.9876 - val_loss: 0.2954 - val_accuracy: 0.9343 - lr: 2.5000e-04

Epoch 78: LearningRateScheduler setting learning rate to 0.0002500000118543308.
Epoch 78/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0609 - accuracy: 0.9842 - val_loss: 0.2825 - val_accuracy: 0.9387 - lr: 2.5000e-04

Epoch 79: LearningRateScheduler setting learning rate to 0.0002500000118545844.
Epoch 79/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0522 - accuracy: 0.9853 - val_loss: 0.3312 - val_accuracy: 0.9307 - lr: 2.5000e-04

Epoch 80: LearningRateScheduler setting learning rate to 0.0002500000118548316.
Epoch 80/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0572 - accuracy: 0.9846 - val_loss: 0.2989 - val_accuracy: 0.9360 - lr: 2.5000e-04

Epoch 81: LearningRateScheduler setting learning rate to 0.0002500000118550727.
Epoch 81/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0548 - accuracy: 0.9859 - val_loss: 0.4023 - val_accuracy: 0.9210 - lr: 2.5000e-04

Epoch 82: LearningRateScheduler setting learning rate to 0.000250000011855308.
Epoch 82/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0709 - accuracy: 0.9812 - val_loss: 0.2938 - val_accuracy: 0.9333 - lr: 2.5000e-04

Epoch 83: LearningRateScheduler setting learning rate to 0.00025000001185553755.
Epoch 83/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0525 - accuracy: 0.9842 - val_loss: 0.2725 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 84: LearningRateScheduler setting learning rate to 0.00025000001185576166.
Epoch 84/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0689 - accuracy: 0.9814 - val_loss: 0.2937 - val_accuracy: 0.9383 - lr: 2.5000e-04

Epoch 85: LearningRateScheduler setting learning rate to 0.00025000001185598045.
Epoch 85/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0664 - accuracy: 0.9825 - val_loss: 0.2991 - val_accuracy: 0.9343 - lr: 2.5000e-04

Epoch 86: LearningRateScheduler setting learning rate to 0.00025000001185619425.
Epoch 86/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0532 - accuracy: 0.9846 - val_loss: 0.2623 - val_accuracy: 0.9410 - lr: 2.5000e-04

Epoch 87: LearningRateScheduler setting learning rate to 0.00025000001185640307.
Epoch 87/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0599 - accuracy: 0.9824 - val_loss: 0.3218 - val_accuracy: 0.9340 - lr: 2.5000e-04

Epoch 88: LearningRateScheduler setting learning rate to 0.0002500000118566071.
Epoch 88/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0481 - accuracy: 0.9858 - val_loss: 0.2852 - val_accuracy: 0.9413 - lr: 2.5000e-04

Epoch 89: LearningRateScheduler setting learning rate to 0.00025000001185680667.
Epoch 89/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0655 - accuracy: 0.9839 - val_loss: 0.2750 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 90: LearningRateScheduler setting learning rate to 0.0002500000118570017.
Epoch 90/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0603 - accuracy: 0.9843 - val_loss: 0.3164 - val_accuracy: 0.9330 - lr: 2.5000e-04

Epoch 91: LearningRateScheduler setting learning rate to 0.0002500000118571925.
Epoch 91/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0443 - accuracy: 0.9888 - val_loss: 0.2720 - val_accuracy: 0.9393 - lr: 2.5000e-04

Epoch 92: LearningRateScheduler setting learning rate to 0.0002500000118573791.
Epoch 92/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0478 - accuracy: 0.9864 - val_loss: 0.3510 - val_accuracy: 0.9293 - lr: 2.5000e-04

Epoch 93: LearningRateScheduler setting learning rate to 0.00025000001185756176.
Epoch 93/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0566 - accuracy: 0.9838 - val_loss: 0.2970 - val_accuracy: 0.9353 - lr: 2.5000e-04

Epoch 94: LearningRateScheduler setting learning rate to 0.0002500000118577405.
Epoch 94/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0561 - accuracy: 0.9846 - val_loss: 0.4237 - val_accuracy: 0.9193 - lr: 2.5000e-04

Epoch 95: LearningRateScheduler setting learning rate to 0.0002500000118579155.
Epoch 95/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0580 - accuracy: 0.9829 - val_loss: 0.2994 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 96: LearningRateScheduler setting learning rate to 0.0002500000118580868.
Epoch 96/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0563 - accuracy: 0.9854 - val_loss: 0.3126 - val_accuracy: 0.9293 - lr: 2.5000e-04

Epoch 97: LearningRateScheduler setting learning rate to 0.0002500000118582546.
Epoch 97/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0421 - accuracy: 0.9884 - val_loss: 0.3309 - val_accuracy: 0.9287 - lr: 2.5000e-04

Epoch 98: LearningRateScheduler setting learning rate to 0.000250000011858419.
Epoch 98/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0451 - accuracy: 0.9889 - val_loss: 0.4132 - val_accuracy: 0.9223 - lr: 2.5000e-04

Epoch 99: LearningRateScheduler setting learning rate to 0.00025000001185858.
Epoch 99/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0533 - accuracy: 0.9867 - val_loss: 0.3135 - val_accuracy: 0.9443 - lr: 2.5000e-04

Epoch 100: LearningRateScheduler setting learning rate to 0.00025000001185873785.
Epoch 100/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0403 - accuracy: 0.9893 - val_loss: 0.3476 - val_accuracy: 0.9383 - lr: 2.5000e-04

Epoch 101: LearningRateScheduler setting learning rate to 0.0002500000118588925.
Epoch 101/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0519 - accuracy: 0.9867 - val_loss: 0.2851 - val_accuracy: 0.9410 - lr: 2.5000e-04

Epoch 102: LearningRateScheduler setting learning rate to 0.0002500000118590442.
Epoch 102/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0658 - accuracy: 0.9834 - val_loss: 0.3326 - val_accuracy: 0.9287 - lr: 2.5000e-04

Epoch 103: LearningRateScheduler setting learning rate to 0.0002500000118591929.
Epoch 103/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0446 - accuracy: 0.9880 - val_loss: 0.3108 - val_accuracy: 0.9350 - lr: 2.5000e-04

Epoch 104: LearningRateScheduler setting learning rate to 0.00025000001185933877.
Epoch 104/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0491 - accuracy: 0.9870 - val_loss: 0.3162 - val_accuracy: 0.9400 - lr: 2.5000e-04

Epoch 105: LearningRateScheduler setting learning rate to 0.0002500000118594819.
Epoch 105/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0464 - accuracy: 0.9880 - val_loss: 0.3336 - val_accuracy: 0.9363 - lr: 2.5000e-04

Epoch 106: LearningRateScheduler setting learning rate to 0.0002500000118596223.
Epoch 106/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0481 - accuracy: 0.9870 - val_loss: 0.2744 - val_accuracy: 0.9440 - lr: 2.5000e-04

Epoch 107: LearningRateScheduler setting learning rate to 0.00025000001185976003.
Epoch 107/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0523 - accuracy: 0.9877 - val_loss: 0.2690 - val_accuracy: 0.9433 - lr: 2.5000e-04

Epoch 108: LearningRateScheduler setting learning rate to 0.00025000001185989523.
Epoch 108/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0467 - accuracy: 0.9879 - val_loss: 0.2862 - val_accuracy: 0.9430 - lr: 2.5000e-04

Epoch 109: LearningRateScheduler setting learning rate to 0.00025000001186002794.
Epoch 109/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0464 - accuracy: 0.9897 - val_loss: 0.2864 - val_accuracy: 0.9407 - lr: 2.5000e-04

Epoch 110: LearningRateScheduler setting learning rate to 0.0002500000118601583.
Epoch 110/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0442 - accuracy: 0.9876 - val_loss: 0.2920 - val_accuracy: 0.9420 - lr: 2.5000e-04

Epoch 111: LearningRateScheduler setting learning rate to 0.00025000001186028625.
Epoch 111/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0497 - accuracy: 0.9869 - val_loss: 0.2904 - val_accuracy: 0.9400 - lr: 2.5000e-04

Epoch 112: LearningRateScheduler setting learning rate to 0.0002500000118604119.
Epoch 112/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0534 - accuracy: 0.9863 - val_loss: 0.2811 - val_accuracy: 0.9430 - lr: 2.5000e-04

Epoch 113: LearningRateScheduler setting learning rate to 0.0002500000118605354.
Epoch 113/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0455 - accuracy: 0.9875 - val_loss: 0.3504 - val_accuracy: 0.9287 - lr: 2.5000e-04

Epoch 114: LearningRateScheduler setting learning rate to 0.00025000001186065667.
Epoch 114/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0519 - accuracy: 0.9862 - val_loss: 0.3545 - val_accuracy: 0.9280 - lr: 2.5000e-04

Epoch 115: LearningRateScheduler setting learning rate to 0.0002500000118607759.
Epoch 115/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0408 - accuracy: 0.9884 - val_loss: 0.2944 - val_accuracy: 0.9413 - lr: 2.5000e-04

Epoch 116: LearningRateScheduler setting learning rate to 0.00025000001186089297.
Epoch 116/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0361 - accuracy: 0.9889 - val_loss: 0.3560 - val_accuracy: 0.9293 - lr: 2.5000e-04

Epoch 117: LearningRateScheduler setting learning rate to 0.0002500000118610081.
Epoch 117/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0439 - accuracy: 0.9900 - val_loss: 0.3357 - val_accuracy: 0.9307 - lr: 2.5000e-04

Epoch 118: LearningRateScheduler setting learning rate to 0.0002500000118611213.
Epoch 118/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0472 - accuracy: 0.9867 - val_loss: 0.3437 - val_accuracy: 0.9270 - lr: 2.5000e-04

Epoch 119: LearningRateScheduler setting learning rate to 0.00025000001186123254.
Epoch 119/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0498 - accuracy: 0.9879 - val_loss: 0.3410 - val_accuracy: 0.9330 - lr: 2.5000e-04

Epoch 120: LearningRateScheduler setting learning rate to 0.00025000001186134205.
Epoch 120/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0532 - accuracy: 0.9870 - val_loss: 0.2977 - val_accuracy: 0.9413 - lr: 2.5000e-04

Epoch 121: LearningRateScheduler setting learning rate to 0.0002500000118614496.
Epoch 121/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0608 - accuracy: 0.9859 - val_loss: 0.3547 - val_accuracy: 0.9340 - lr: 2.5000e-04

Epoch 122: LearningRateScheduler setting learning rate to 0.00025000001186155547.
Epoch 122/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0370 - accuracy: 0.9907 - val_loss: 0.3107 - val_accuracy: 0.9393 - lr: 2.5000e-04

Epoch 123: LearningRateScheduler setting learning rate to 0.0002500000118616596.
Epoch 123/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0406 - accuracy: 0.9903 - val_loss: 0.3647 - val_accuracy: 0.9307 - lr: 2.5000e-04

Epoch 124: LearningRateScheduler setting learning rate to 0.000250000011861762.
Epoch 124/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0537 - accuracy: 0.9854 - val_loss: 0.2992 - val_accuracy: 0.9443 - lr: 2.5000e-04

Epoch 125: LearningRateScheduler setting learning rate to 0.00025000001186186284.
Epoch 125/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0443 - accuracy: 0.9881 - val_loss: 0.3815 - val_accuracy: 0.9317 - lr: 2.5000e-04

Epoch 126: LearningRateScheduler setting learning rate to 0.00025000001186196205.
Epoch 126/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0406 - accuracy: 0.9884 - val_loss: 0.3362 - val_accuracy: 0.9403 - lr: 2.5000e-04

Epoch 127: LearningRateScheduler setting learning rate to 0.0002500000118620597.
Epoch 127/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0415 - accuracy: 0.9894 - val_loss: 0.3148 - val_accuracy: 0.9413 - lr: 2.5000e-04

Epoch 128: LearningRateScheduler setting learning rate to 0.0002500000118621558.
Epoch 128/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0380 - accuracy: 0.9908 - val_loss: 0.3223 - val_accuracy: 0.9380 - lr: 2.5000e-04

Epoch 129: LearningRateScheduler setting learning rate to 0.0002500000118622504.
Epoch 129/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0454 - accuracy: 0.9875 - val_loss: 0.3327 - val_accuracy: 0.9387 - lr: 2.5000e-04

Epoch 130: LearningRateScheduler setting learning rate to 0.00025000001186234363.
Epoch 130/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0532 - accuracy: 0.9869 - val_loss: 0.3248 - val_accuracy: 0.9353 - lr: 2.5000e-04

Epoch 131: LearningRateScheduler setting learning rate to 0.00025000001186243536.
Epoch 131/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0347 - accuracy: 0.9906 - val_loss: 0.3764 - val_accuracy: 0.9277 - lr: 2.5000e-04

Epoch 132: LearningRateScheduler setting learning rate to 0.00025000001186252567.
Epoch 132/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0539 - accuracy: 0.9866 - val_loss: 0.2790 - val_accuracy: 0.9397 - lr: 2.5000e-04

Epoch 133: LearningRateScheduler setting learning rate to 0.00025000001186261474.
Epoch 133/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0384 - accuracy: 0.9886 - val_loss: 0.2985 - val_accuracy: 0.9427 - lr: 2.5000e-04

Epoch 134: LearningRateScheduler setting learning rate to 0.0002500000118627024.
Epoch 134/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0297 - accuracy: 0.9912 - val_loss: 0.3746 - val_accuracy: 0.9360 - lr: 2.5000e-04

Epoch 135: LearningRateScheduler setting learning rate to 0.00025000001186278875.
Epoch 135/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0517 - accuracy: 0.9880 - val_loss: 0.3710 - val_accuracy: 0.9307 - lr: 2.5000e-04

Epoch 136: LearningRateScheduler setting learning rate to 0.00025000001186287386.
Epoch 136/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0474 - accuracy: 0.9872 - val_loss: 0.3818 - val_accuracy: 0.9303 - lr: 2.5000e-04

Epoch 137: LearningRateScheduler setting learning rate to 0.0002500000118629577.
Epoch 137/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0389 - accuracy: 0.9896 - val_loss: 0.2997 - val_accuracy: 0.9423 - lr: 2.5000e-04

Epoch 138: LearningRateScheduler setting learning rate to 0.0002500000118630404.
Epoch 138/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0485 - accuracy: 0.9885 - val_loss: 0.3348 - val_accuracy: 0.9427 - lr: 2.5000e-04

Epoch 139: LearningRateScheduler setting learning rate to 0.0002500000118631218.
Epoch 139/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0477 - accuracy: 0.9883 - val_loss: 0.5134 - val_accuracy: 0.9077 - lr: 2.5000e-04

Epoch 140: LearningRateScheduler setting learning rate to 0.0002500000118632021.
Epoch 140/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0466 - accuracy: 0.9886 - val_loss: 0.2966 - val_accuracy: 0.9380 - lr: 2.5000e-04

Epoch 141: LearningRateScheduler setting learning rate to 0.00025000001186328125.
Epoch 141/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0421 - accuracy: 0.9891 - val_loss: 0.3156 - val_accuracy: 0.9377 - lr: 2.5000e-04

Epoch 142: LearningRateScheduler setting learning rate to 0.0002500000118633593.
Epoch 142/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0445 - accuracy: 0.9883 - val_loss: 0.2900 - val_accuracy: 0.9420 - lr: 2.5000e-04

Epoch 143: LearningRateScheduler setting learning rate to 0.00025000001186343624.
Epoch 143/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0494 - accuracy: 0.9877 - val_loss: 0.3125 - val_accuracy: 0.9410 - lr: 2.5000e-04

Epoch 144: LearningRateScheduler setting learning rate to 0.00025000001186351213.
Epoch 144/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0326 - accuracy: 0.9917 - val_loss: 0.3096 - val_accuracy: 0.9423 - lr: 2.5000e-04

Epoch 145: LearningRateScheduler setting learning rate to 0.00025000001186358694.
Epoch 145/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0299 - accuracy: 0.9920 - val_loss: 0.3036 - val_accuracy: 0.9447 - lr: 2.5000e-04

Epoch 146: LearningRateScheduler setting learning rate to 0.0002500000118636608.
Epoch 146/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0371 - accuracy: 0.9900 - val_loss: 0.4279 - val_accuracy: 0.9317 - lr: 2.5000e-04

Epoch 147: LearningRateScheduler setting learning rate to 0.0002500000118637335.
Epoch 147/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0445 - accuracy: 0.9896 - val_loss: 0.3733 - val_accuracy: 0.9360 - lr: 2.5000e-04

Epoch 148: LearningRateScheduler setting learning rate to 0.0002500000118638054.
Epoch 148/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0327 - accuracy: 0.9906 - val_loss: 0.3434 - val_accuracy: 0.9413 - lr: 2.5000e-04

Epoch 149: LearningRateScheduler setting learning rate to 0.00025000001186387626.
Epoch 149/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0428 - accuracy: 0.9898 - val_loss: 0.3108 - val_accuracy: 0.9380 - lr: 2.5000e-04

Epoch 150: LearningRateScheduler setting learning rate to 0.00025000001186394614.
Epoch 150/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0468 - accuracy: 0.9877 - val_loss: 0.2926 - val_accuracy: 0.9447 - lr: 2.5000e-04

Epoch 151: LearningRateScheduler setting learning rate to 0.00025000001186401515.
Epoch 151/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0466 - accuracy: 0.9887 - val_loss: 0.3107 - val_accuracy: 0.9390 - lr: 2.5000e-04

Epoch 152: LearningRateScheduler setting learning rate to 0.00025000001186408323.
Epoch 152/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0233 - accuracy: 0.9938 - val_loss: 0.3361 - val_accuracy: 0.9433 - lr: 2.5000e-04

Epoch 153: LearningRateScheduler setting learning rate to 0.0002500000118641504.
Epoch 153/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0590 - accuracy: 0.9855 - val_loss: 0.3199 - val_accuracy: 0.9383 - lr: 2.5000e-04

Epoch 154: LearningRateScheduler setting learning rate to 0.0002500000118642167.
Epoch 154/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0371 - accuracy: 0.9903 - val_loss: 0.3520 - val_accuracy: 0.9363 - lr: 2.5000e-04

Epoch 155: LearningRateScheduler setting learning rate to 0.0002500000118642822.
Epoch 155/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0413 - accuracy: 0.9896 - val_loss: 0.2772 - val_accuracy: 0.9447 - lr: 2.5000e-04

Epoch 156: LearningRateScheduler setting learning rate to 0.0002500000118643468.
Epoch 156/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0548 - accuracy: 0.9883 - val_loss: 0.2966 - val_accuracy: 0.9447 - lr: 2.5000e-04

Epoch 157: LearningRateScheduler setting learning rate to 0.0002500000118644106.
Epoch 157/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0316 - accuracy: 0.9928 - val_loss: 0.3235 - val_accuracy: 0.9373 - lr: 2.5000e-04

Epoch 158: LearningRateScheduler setting learning rate to 0.00025000001186447355.
Epoch 158/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0505 - accuracy: 0.9891 - val_loss: 0.3202 - val_accuracy: 0.9407 - lr: 2.5000e-04

Epoch 159: LearningRateScheduler setting learning rate to 0.0002500000118645358.
Epoch 159/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0432 - accuracy: 0.9891 - val_loss: 0.3726 - val_accuracy: 0.9407 - lr: 2.5000e-04

Epoch 160: LearningRateScheduler setting learning rate to 0.0002500000118645972.
Epoch 160/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0509 - accuracy: 0.9868 - val_loss: 0.3186 - val_accuracy: 0.9407 - lr: 2.5000e-04

Epoch 161: LearningRateScheduler setting learning rate to 0.00025000001186465786.
Epoch 161/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0334 - accuracy: 0.9924 - val_loss: 0.3530 - val_accuracy: 0.9403 - lr: 2.5000e-04

Epoch 162: LearningRateScheduler setting learning rate to 0.00025000001186471776.
Epoch 162/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0290 - accuracy: 0.9918 - val_loss: 0.3228 - val_accuracy: 0.9480 - lr: 2.5000e-04

Epoch 163: LearningRateScheduler setting learning rate to 0.00025000001186477696.
Epoch 163/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0486 - accuracy: 0.9887 - val_loss: 0.3377 - val_accuracy: 0.9410 - lr: 2.5000e-04

Epoch 164: LearningRateScheduler setting learning rate to 0.0002500000118648354.
Epoch 164/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0423 - accuracy: 0.9888 - val_loss: 0.3361 - val_accuracy: 0.9393 - lr: 2.5000e-04

Epoch 165: LearningRateScheduler setting learning rate to 0.00025000001186489313.
Epoch 165/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0492 - accuracy: 0.9873 - val_loss: 0.2811 - val_accuracy: 0.9450 - lr: 2.5000e-04

Epoch 166: LearningRateScheduler setting learning rate to 0.00025000001186495016.
Epoch 166/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0311 - accuracy: 0.9904 - val_loss: 0.2795 - val_accuracy: 0.9453 - lr: 2.5000e-04

Epoch 167: LearningRateScheduler setting learning rate to 0.00025000001186500654.
Epoch 167/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0340 - accuracy: 0.9912 - val_loss: 0.3625 - val_accuracy: 0.9400 - lr: 2.5000e-04

Epoch 168: LearningRateScheduler setting learning rate to 0.0002500000118650622.
Epoch 168/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0461 - accuracy: 0.9888 - val_loss: 0.3355 - val_accuracy: 0.9423 - lr: 2.5000e-04

Epoch 169: LearningRateScheduler setting learning rate to 0.0002500000118651173.
Epoch 169/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0450 - accuracy: 0.9911 - val_loss: 0.3685 - val_accuracy: 0.9327 - lr: 2.5000e-04

Epoch 170: LearningRateScheduler setting learning rate to 0.0002500000118651716.
Epoch 170/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0477 - accuracy: 0.9897 - val_loss: 0.3131 - val_accuracy: 0.9450 - lr: 2.5000e-04

Epoch 171: LearningRateScheduler setting learning rate to 0.0002500000118652254.
Epoch 171/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0429 - accuracy: 0.9905 - val_loss: 0.3095 - val_accuracy: 0.9413 - lr: 2.5000e-04

Epoch 172: LearningRateScheduler setting learning rate to 0.0002500000118652785.
Epoch 172/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0371 - accuracy: 0.9895 - val_loss: 0.3286 - val_accuracy: 0.9450 - lr: 2.5000e-04

Epoch 173: LearningRateScheduler setting learning rate to 0.00025000001186533104.
Epoch 173/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0245 - accuracy: 0.9936 - val_loss: 0.4419 - val_accuracy: 0.9243 - lr: 2.5000e-04

Epoch 174: LearningRateScheduler setting learning rate to 0.000250000011865383.
Epoch 174/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0655 - accuracy: 0.9878 - val_loss: 0.3386 - val_accuracy: 0.9370 - lr: 2.5000e-04

Epoch 175: LearningRateScheduler setting learning rate to 0.00025000001186543426.
Epoch 175/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0395 - accuracy: 0.9903 - val_loss: 0.3269 - val_accuracy: 0.9373 - lr: 2.5000e-04

Epoch 176: LearningRateScheduler setting learning rate to 0.000250000011865485.
Epoch 176/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0435 - accuracy: 0.9910 - val_loss: 0.3156 - val_accuracy: 0.9403 - lr: 2.5000e-04

Epoch 177: LearningRateScheduler setting learning rate to 0.00025000001186553514.
Epoch 177/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0348 - accuracy: 0.9922 - val_loss: 0.3571 - val_accuracy: 0.9397 - lr: 2.5000e-04

Epoch 178: LearningRateScheduler setting learning rate to 0.00025000001186558475.
Epoch 178/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0392 - accuracy: 0.9904 - val_loss: 0.3630 - val_accuracy: 0.9410 - lr: 2.5000e-04

Epoch 179: LearningRateScheduler setting learning rate to 0.00025000001186563375.
Epoch 179/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0396 - accuracy: 0.9900 - val_loss: 0.2991 - val_accuracy: 0.9433 - lr: 2.5000e-04

Epoch 180: LearningRateScheduler setting learning rate to 0.00025000001186568227.
Epoch 180/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0373 - accuracy: 0.9905 - val_loss: 0.3127 - val_accuracy: 0.9423 - lr: 2.5000e-04

Epoch 181: LearningRateScheduler setting learning rate to 0.00025000001186573025.
Epoch 181/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0504 - accuracy: 0.9885 - val_loss: 0.3218 - val_accuracy: 0.9400 - lr: 2.5000e-04

Epoch 182: LearningRateScheduler setting learning rate to 0.0002500000118657776.
Epoch 182/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0441 - accuracy: 0.9899 - val_loss: 0.3250 - val_accuracy: 0.9407 - lr: 2.5000e-04

Epoch 183: LearningRateScheduler setting learning rate to 0.0002500000118658246.
Epoch 183/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0328 - accuracy: 0.9906 - val_loss: 0.3365 - val_accuracy: 0.9433 - lr: 2.5000e-04

Epoch 184: LearningRateScheduler setting learning rate to 0.000250000011865871.
Epoch 184/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0398 - accuracy: 0.9904 - val_loss: 0.3822 - val_accuracy: 0.9337 - lr: 2.5000e-04

Epoch 185: LearningRateScheduler setting learning rate to 0.00025000001186591684.
Epoch 185/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0426 - accuracy: 0.9896 - val_loss: 0.3797 - val_accuracy: 0.9330 - lr: 2.5000e-04

Epoch 186: LearningRateScheduler setting learning rate to 0.00025000001186596227.
Epoch 186/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0386 - accuracy: 0.9924 - val_loss: 0.4201 - val_accuracy: 0.9343 - lr: 2.5000e-04

Epoch 187: LearningRateScheduler setting learning rate to 0.0002500000118660072.
Epoch 187/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0444 - accuracy: 0.9894 - val_loss: 0.3256 - val_accuracy: 0.9440 - lr: 2.5000e-04

Epoch 188: LearningRateScheduler setting learning rate to 0.00025000001186605166.
Epoch 188/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0444 - accuracy: 0.9903 - val_loss: 0.3159 - val_accuracy: 0.9360 - lr: 2.5000e-04

Epoch 189: LearningRateScheduler setting learning rate to 0.0002500000118660956.
Epoch 189/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0421 - accuracy: 0.9895 - val_loss: 0.3406 - val_accuracy: 0.9387 - lr: 2.5000e-04

Epoch 190: LearningRateScheduler setting learning rate to 0.00025000001186613915.
Epoch 190/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0387 - accuracy: 0.9912 - val_loss: 0.3140 - val_accuracy: 0.9413 - lr: 2.5000e-04

Epoch 191: LearningRateScheduler setting learning rate to 0.0002500000118661822.
Epoch 191/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0374 - accuracy: 0.9910 - val_loss: 0.3965 - val_accuracy: 0.9230 - lr: 2.5000e-04

Epoch 192: LearningRateScheduler setting learning rate to 0.0002500000118662248.
Epoch 192/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0494 - accuracy: 0.9885 - val_loss: 0.3245 - val_accuracy: 0.9430 - lr: 2.5000e-04

Epoch 193: LearningRateScheduler setting learning rate to 0.000250000011866267.
Epoch 193/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0355 - accuracy: 0.9914 - val_loss: 0.3270 - val_accuracy: 0.9407 - lr: 2.5000e-04

Epoch 194: LearningRateScheduler setting learning rate to 0.0002500000118663087.
Epoch 194/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0472 - accuracy: 0.9888 - val_loss: 0.3411 - val_accuracy: 0.9337 - lr: 2.5000e-04

Epoch 195: LearningRateScheduler setting learning rate to 0.00025000001186635003.
Epoch 195/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0364 - accuracy: 0.9910 - val_loss: 0.3462 - val_accuracy: 0.9367 - lr: 2.5000e-04

Epoch 196: LearningRateScheduler setting learning rate to 0.00025000001186639085.
Epoch 196/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0345 - accuracy: 0.9907 - val_loss: 0.3962 - val_accuracy: 0.9383 - lr: 2.5000e-04

Epoch 197: LearningRateScheduler setting learning rate to 0.00025000001186643135.
Epoch 197/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0288 - accuracy: 0.9940 - val_loss: 0.3572 - val_accuracy: 0.9467 - lr: 2.5000e-04

Epoch 198: LearningRateScheduler setting learning rate to 0.0002500000118664714.
Epoch 198/200
903/903 [==============================] - 8s 8ms/step - loss: 0.0384 - accuracy: 0.9925 - val_loss: 0.3285 - val_accuracy: 0.9457 - lr: 2.5000e-04

Epoch 199: LearningRateScheduler setting learning rate to 0.00025000001186651103.
Epoch 199/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0368 - accuracy: 0.9911 - val_loss: 0.2799 - val_accuracy: 0.9420 - lr: 2.5000e-04

Epoch 200: LearningRateScheduler setting learning rate to 0.00025000001186655034.
Epoch 200/200
903/903 [==============================] - 7s 8ms/step - loss: 0.0493 - accuracy: 0.9893 - val_loss: 0.3344 - val_accuracy: 0.9407 - lr: 2.5000e-04

Observations

• Learning rate once again did not change much but the validation accuracy still managed to improve all the way to 94%.

Conv2D_31_lr.summary()

Model: "Conv2D_31V1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalised_data (Sequential  (None, None, None, 1)    0         
 )                                                               
                                                                 
 conv2d (Conv2D)             (None, 31, 31, 32)        320       
                                                                 
 max_pooling2d (MaxPooling2D  (None, 15, 15, 32)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 15, 15, 64)        18496     
                                                                 
 conv2d_2 (Conv2D)           (None, 15, 15, 128)       73856     
                                                                 
 conv2d_3 (Conv2D)           (None, 15, 15, 128)       147584    
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 7, 7, 128)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 6272)              0         
                                                                 
 dropout (Dropout)           (None, 6272)              0         
                                                                 
 dense (Dense)               (None, 512)               3211776   
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_1 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               65792     
                                                                 
 dropout_3 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 15)                3855      
                                                                 
=================================================================
Total params: 3,653,007
Trainable params: 3,653,007
Non-trainable params: 0
_________________________________________________________________

plot_learning_curve(Conv2D_31_lr_history.history)

Observations

• Validation loss and accuracy are quite close to the training loss and accuracy, achieving close to 0.9+ accuracy, loss is also kept as low as 0.5+

Conv2D_31_lr.evaluate(test_data_31.batch(10))

300/300 [==============================] - 2s 5ms/step - loss: 0.2863 - accuracy: 0.9433

[0.28634753823280334, 0.9433333277702332]

Observations

• Might have to decrease the steps for the learning rate to have a more drastic change
• Test accuracy is still 90+ which shows how accurate this model is.

Regularization - Reduces overfitting and improve generalisation performance of the model

## Using L2 Regularization for 31 x 31 images
tf.keras.backend.clear_session()

Conv2D_31_L2 = Sequential(name="Conv2D_31_L2",
    layers = [
        normalised_data,
        Conv2D(32, (3, 3),input_shape=(31, 31 ,1), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),
        Conv2D(64, (3, 3), activation='relu', padding='same'),
        Conv2D(128, (3, 3), activation='relu', padding='same'),
        Conv2D(128, (3, 3), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.5),

        Dense(512, activation='relu', kernel_regularizer=l2(0.01)),
        Dropout(0.5),
        Dense(256, activation='relu', kernel_regularizer=l2(0.01)),
        Dropout(0.5),
        Dense(256, activation='relu', kernel_regularizer=l2(0.01)),
        Dropout(0.5),

        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.0001)
Conv2D_31_L2.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_31_L2.build(input_shape=(None, 31, 31, 1))

Conv2D_31_L2_history = Conv2D_31_L2.fit(
    train_data_31.batch(10), 
    epochs=200,
    validation_data=val_data_31.batch(10)
    )

Epoch 1/200
903/903 [==============================] - 8s 8ms/step - loss: 7.3834 - accuracy: 0.0912 - val_loss: 4.1177 - val_accuracy: 0.0667
Epoch 2/200
903/903 [==============================] - 8s 8ms/step - loss: 3.3857 - accuracy: 0.1459 - val_loss: 3.0769 - val_accuracy: 0.1787
Epoch 3/200
903/903 [==============================] - 8s 8ms/step - loss: 2.8199 - accuracy: 0.2111 - val_loss: 2.6438 - val_accuracy: 0.3187
Epoch 4/200
903/903 [==============================] - 8s 8ms/step - loss: 2.4926 - accuracy: 0.3188 - val_loss: 2.3643 - val_accuracy: 0.3783
Epoch 5/200
903/903 [==============================] - 8s 8ms/step - loss: 2.2984 - accuracy: 0.3816 - val_loss: 2.1836 - val_accuracy: 0.4220
Epoch 6/200
903/903 [==============================] - 8s 8ms/step - loss: 2.1713 - accuracy: 0.4251 - val_loss: 2.0838 - val_accuracy: 0.4353
Epoch 7/200
903/903 [==============================] - 8s 8ms/step - loss: 2.0778 - accuracy: 0.4520 - val_loss: 1.9781 - val_accuracy: 0.4763
Epoch 8/200
903/903 [==============================] - 8s 8ms/step - loss: 2.0026 - accuracy: 0.4853 - val_loss: 1.8783 - val_accuracy: 0.5080
Epoch 9/200
903/903 [==============================] - 8s 8ms/step - loss: 1.9217 - accuracy: 0.5148 - val_loss: 1.8615 - val_accuracy: 0.4973
Epoch 10/200
903/903 [==============================] - 8s 8ms/step - loss: 1.8630 - accuracy: 0.5309 - val_loss: 1.7839 - val_accuracy: 0.5377
Epoch 11/200
903/903 [==============================] - 8s 9ms/step - loss: 1.8115 - accuracy: 0.5504 - val_loss: 1.6970 - val_accuracy: 0.5610
Epoch 12/200
903/903 [==============================] - 8s 8ms/step - loss: 1.7894 - accuracy: 0.5626 - val_loss: 1.6807 - val_accuracy: 0.5763
Epoch 13/200
903/903 [==============================] - 8s 8ms/step - loss: 1.7276 - accuracy: 0.5813 - val_loss: 1.6335 - val_accuracy: 0.6117
Epoch 14/200
903/903 [==============================] - 8s 9ms/step - loss: 1.7104 - accuracy: 0.5919 - val_loss: 1.6185 - val_accuracy: 0.6067
Epoch 15/200
903/903 [==============================] - 8s 8ms/step - loss: 1.6736 - accuracy: 0.6068 - val_loss: 1.5933 - val_accuracy: 0.6160
Epoch 16/200
903/903 [==============================] - 8s 8ms/step - loss: 1.6469 - accuracy: 0.6208 - val_loss: 1.4988 - val_accuracy: 0.6663
Epoch 17/200
903/903 [==============================] - 8s 8ms/step - loss: 1.6213 - accuracy: 0.6334 - val_loss: 1.5025 - val_accuracy: 0.6703
Epoch 18/200
903/903 [==============================] - 7s 8ms/step - loss: 1.5836 - accuracy: 0.6455 - val_loss: 1.4667 - val_accuracy: 0.6737
Epoch 19/200
903/903 [==============================] - 8s 8ms/step - loss: 1.5393 - accuracy: 0.6606 - val_loss: 1.4175 - val_accuracy: 0.7053
Epoch 20/200
903/903 [==============================] - 8s 8ms/step - loss: 1.5305 - accuracy: 0.6665 - val_loss: 1.4993 - val_accuracy: 0.6753
Epoch 21/200
903/903 [==============================] - 8s 8ms/step - loss: 1.5183 - accuracy: 0.6810 - val_loss: 1.3791 - val_accuracy: 0.7293
Epoch 22/200
903/903 [==============================] - 7s 8ms/step - loss: 1.5059 - accuracy: 0.6848 - val_loss: 1.4013 - val_accuracy: 0.7157
Epoch 23/200
903/903 [==============================] - 7s 8ms/step - loss: 1.4684 - accuracy: 0.7051 - val_loss: 1.3719 - val_accuracy: 0.7357
Epoch 24/200
903/903 [==============================] - 8s 8ms/step - loss: 1.4496 - accuracy: 0.7101 - val_loss: 1.3630 - val_accuracy: 0.7397
Epoch 25/200
903/903 [==============================] - 8s 8ms/step - loss: 1.4150 - accuracy: 0.7209 - val_loss: 1.2979 - val_accuracy: 0.7737
Epoch 26/200
903/903 [==============================] - 8s 9ms/step - loss: 1.4161 - accuracy: 0.7298 - val_loss: 1.3427 - val_accuracy: 0.7460
Epoch 27/200
903/903 [==============================] - 8s 8ms/step - loss: 1.4022 - accuracy: 0.7365 - val_loss: 1.2923 - val_accuracy: 0.7710
Epoch 28/200
903/903 [==============================] - 8s 8ms/step - loss: 1.3858 - accuracy: 0.7384 - val_loss: 1.2834 - val_accuracy: 0.7860
Epoch 29/200
903/903 [==============================] - 7s 8ms/step - loss: 1.3531 - accuracy: 0.7521 - val_loss: 1.2587 - val_accuracy: 0.7887
Epoch 30/200
903/903 [==============================] - 7s 8ms/step - loss: 1.3455 - accuracy: 0.7472 - val_loss: 1.2312 - val_accuracy: 0.8033
Epoch 31/200
903/903 [==============================] - 8s 8ms/step - loss: 1.3305 - accuracy: 0.7679 - val_loss: 1.2141 - val_accuracy: 0.8063
Epoch 32/200
903/903 [==============================] - 7s 8ms/step - loss: 1.3005 - accuracy: 0.7706 - val_loss: 1.1895 - val_accuracy: 0.8140
Epoch 33/200
903/903 [==============================] - 8s 8ms/step - loss: 1.3207 - accuracy: 0.7646 - val_loss: 1.2247 - val_accuracy: 0.8020
Epoch 34/200
903/903 [==============================] - 8s 8ms/step - loss: 1.2781 - accuracy: 0.7829 - val_loss: 1.2070 - val_accuracy: 0.8060
Epoch 35/200
903/903 [==============================] - 8s 8ms/step - loss: 1.2676 - accuracy: 0.7875 - val_loss: 1.1963 - val_accuracy: 0.8147
Epoch 36/200
903/903 [==============================] - 8s 8ms/step - loss: 1.2621 - accuracy: 0.7870 - val_loss: 1.1696 - val_accuracy: 0.8197
Epoch 37/200
903/903 [==============================] - 8s 8ms/step - loss: 1.2542 - accuracy: 0.7934 - val_loss: 1.1746 - val_accuracy: 0.8287
Epoch 38/200
903/903 [==============================] - 8s 8ms/step - loss: 1.2401 - accuracy: 0.7944 - val_loss: 1.1946 - val_accuracy: 0.8173
Epoch 39/200
903/903 [==============================] - 8s 8ms/step - loss: 1.2274 - accuracy: 0.8021 - val_loss: 1.1288 - val_accuracy: 0.8413
Epoch 40/200
903/903 [==============================] - 8s 8ms/step - loss: 1.2194 - accuracy: 0.8099 - val_loss: 1.1490 - val_accuracy: 0.8330
Epoch 41/200
903/903 [==============================] - 8s 8ms/step - loss: 1.2233 - accuracy: 0.8005 - val_loss: 1.2325 - val_accuracy: 0.8167
Epoch 42/200
903/903 [==============================] - 7s 8ms/step - loss: 1.2085 - accuracy: 0.8170 - val_loss: 1.1967 - val_accuracy: 0.8223
Epoch 43/200
903/903 [==============================] - 8s 8ms/step - loss: 1.2107 - accuracy: 0.8144 - val_loss: 1.1421 - val_accuracy: 0.8407
Epoch 44/200
903/903 [==============================] - 8s 8ms/step - loss: 1.1963 - accuracy: 0.8154 - val_loss: 1.1126 - val_accuracy: 0.8490
Epoch 45/200
903/903 [==============================] - 8s 8ms/step - loss: 1.1838 - accuracy: 0.8207 - val_loss: 1.1115 - val_accuracy: 0.8487
Epoch 46/200
903/903 [==============================] - 8s 9ms/step - loss: 1.1678 - accuracy: 0.8238 - val_loss: 1.1010 - val_accuracy: 0.8547
Epoch 47/200
903/903 [==============================] - 8s 9ms/step - loss: 1.1725 - accuracy: 0.8303 - val_loss: 1.1414 - val_accuracy: 0.8377
Epoch 48/200
903/903 [==============================] - 8s 8ms/step - loss: 1.1676 - accuracy: 0.8268 - val_loss: 1.0959 - val_accuracy: 0.8530
Epoch 49/200
903/903 [==============================] - 8s 8ms/step - loss: 1.1408 - accuracy: 0.8326 - val_loss: 1.0933 - val_accuracy: 0.8557
Epoch 50/200
903/903 [==============================] - 8s 8ms/step - loss: 1.1467 - accuracy: 0.8304 - val_loss: 1.0744 - val_accuracy: 0.8597
Epoch 51/200
903/903 [==============================] - 8s 8ms/step - loss: 1.1156 - accuracy: 0.8460 - val_loss: 1.0493 - val_accuracy: 0.8713
Epoch 52/200
903/903 [==============================] - 8s 8ms/step - loss: 1.1349 - accuracy: 0.8385 - val_loss: 1.0959 - val_accuracy: 0.8487
Epoch 53/200
903/903 [==============================] - 8s 8ms/step - loss: 1.1084 - accuracy: 0.8463 - val_loss: 1.1291 - val_accuracy: 0.8470
Epoch 54/200
903/903 [==============================] - 8s 8ms/step - loss: 1.1190 - accuracy: 0.8448 - val_loss: 1.0475 - val_accuracy: 0.8690
Epoch 55/200
903/903 [==============================] - 8s 8ms/step - loss: 1.1161 - accuracy: 0.8418 - val_loss: 1.0786 - val_accuracy: 0.8653
Epoch 56/200
903/903 [==============================] - 7s 8ms/step - loss: 1.1069 - accuracy: 0.8468 - val_loss: 1.0650 - val_accuracy: 0.8640
Epoch 57/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0888 - accuracy: 0.8546 - val_loss: 1.0499 - val_accuracy: 0.8707
Epoch 58/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0816 - accuracy: 0.8562 - val_loss: 1.0310 - val_accuracy: 0.8753
Epoch 59/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0690 - accuracy: 0.8598 - val_loss: 1.0323 - val_accuracy: 0.8723
Epoch 60/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0594 - accuracy: 0.8597 - val_loss: 1.0357 - val_accuracy: 0.8730
Epoch 61/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0396 - accuracy: 0.8721 - val_loss: 1.0195 - val_accuracy: 0.8730
Epoch 62/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0692 - accuracy: 0.8533 - val_loss: 1.0566 - val_accuracy: 0.8637
Epoch 63/200
903/903 [==============================] - 7s 8ms/step - loss: 1.0552 - accuracy: 0.8636 - val_loss: 1.0152 - val_accuracy: 0.8733
Epoch 64/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0551 - accuracy: 0.8683 - val_loss: 1.0126 - val_accuracy: 0.8750
Epoch 65/200
903/903 [==============================] - 7s 8ms/step - loss: 1.0371 - accuracy: 0.8646 - val_loss: 1.0060 - val_accuracy: 0.8807
Epoch 66/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0236 - accuracy: 0.8739 - val_loss: 1.0204 - val_accuracy: 0.8773
Epoch 67/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0392 - accuracy: 0.8653 - val_loss: 1.0110 - val_accuracy: 0.8797
Epoch 68/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0263 - accuracy: 0.8712 - val_loss: 1.0161 - val_accuracy: 0.8770
Epoch 69/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0138 - accuracy: 0.8748 - val_loss: 1.0054 - val_accuracy: 0.8803
Epoch 70/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0006 - accuracy: 0.8787 - val_loss: 1.0123 - val_accuracy: 0.8743
Epoch 71/200
903/903 [==============================] - 8s 8ms/step - loss: 1.0199 - accuracy: 0.8716 - val_loss: 0.9994 - val_accuracy: 0.8867
Epoch 72/200
903/903 [==============================] - 7s 8ms/step - loss: 1.0092 - accuracy: 0.8746 - val_loss: 1.0020 - val_accuracy: 0.8863
Epoch 73/200
903/903 [==============================] - 7s 8ms/step - loss: 1.0155 - accuracy: 0.8739 - val_loss: 1.0232 - val_accuracy: 0.8677
Epoch 74/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9907 - accuracy: 0.8814 - val_loss: 0.9940 - val_accuracy: 0.8870
Epoch 75/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9940 - accuracy: 0.8821 - val_loss: 0.9955 - val_accuracy: 0.8790
Epoch 76/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9906 - accuracy: 0.8815 - val_loss: 0.9946 - val_accuracy: 0.8797
Epoch 77/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9843 - accuracy: 0.8821 - val_loss: 0.9739 - val_accuracy: 0.8887
Epoch 78/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9897 - accuracy: 0.8799 - val_loss: 0.9881 - val_accuracy: 0.8793
Epoch 79/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9791 - accuracy: 0.8815 - val_loss: 1.0014 - val_accuracy: 0.8810
Epoch 80/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9537 - accuracy: 0.8875 - val_loss: 0.9774 - val_accuracy: 0.8820
Epoch 81/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9719 - accuracy: 0.8848 - val_loss: 0.9542 - val_accuracy: 0.8893
Epoch 82/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9623 - accuracy: 0.8890 - val_loss: 0.9595 - val_accuracy: 0.8897
Epoch 83/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9712 - accuracy: 0.8829 - val_loss: 0.9668 - val_accuracy: 0.8917
Epoch 84/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9529 - accuracy: 0.8900 - val_loss: 1.0081 - val_accuracy: 0.8837
Epoch 85/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9616 - accuracy: 0.8899 - val_loss: 0.9438 - val_accuracy: 0.8953
Epoch 86/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9555 - accuracy: 0.8905 - val_loss: 0.9596 - val_accuracy: 0.8910
Epoch 87/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9577 - accuracy: 0.8890 - val_loss: 0.9427 - val_accuracy: 0.8917
Epoch 88/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9414 - accuracy: 0.8942 - val_loss: 0.9496 - val_accuracy: 0.8970
Epoch 89/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9397 - accuracy: 0.8953 - val_loss: 0.9313 - val_accuracy: 0.8960
Epoch 90/200
903/903 [==============================] - 8s 9ms/step - loss: 0.9446 - accuracy: 0.8951 - val_loss: 0.9278 - val_accuracy: 0.9003
Epoch 91/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9415 - accuracy: 0.8950 - val_loss: 0.9931 - val_accuracy: 0.8783
Epoch 92/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9258 - accuracy: 0.9018 - val_loss: 0.9514 - val_accuracy: 0.8923
Epoch 93/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9152 - accuracy: 0.8968 - val_loss: 0.9217 - val_accuracy: 0.9053
Epoch 94/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9228 - accuracy: 0.8968 - val_loss: 0.9448 - val_accuracy: 0.8953
Epoch 95/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9203 - accuracy: 0.8985 - val_loss: 0.9229 - val_accuracy: 0.8987
Epoch 96/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9173 - accuracy: 0.9023 - val_loss: 0.9173 - val_accuracy: 0.8990
Epoch 97/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9059 - accuracy: 0.9049 - val_loss: 0.9152 - val_accuracy: 0.9033
Epoch 98/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9058 - accuracy: 0.8994 - val_loss: 0.9615 - val_accuracy: 0.8897
Epoch 99/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9045 - accuracy: 0.9018 - val_loss: 0.9130 - val_accuracy: 0.8990
Epoch 100/200
903/903 [==============================] - 8s 8ms/step - loss: 0.9064 - accuracy: 0.9029 - val_loss: 0.9159 - val_accuracy: 0.9013
Epoch 101/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9200 - accuracy: 0.8975 - val_loss: 0.9429 - val_accuracy: 0.8897
Epoch 102/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8865 - accuracy: 0.9056 - val_loss: 0.9472 - val_accuracy: 0.8897
Epoch 103/200
903/903 [==============================] - 7s 8ms/step - loss: 0.9012 - accuracy: 0.8995 - val_loss: 0.9219 - val_accuracy: 0.8950
Epoch 104/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8906 - accuracy: 0.9042 - val_loss: 0.9131 - val_accuracy: 0.8997
Epoch 105/200
903/903 [==============================] - 8s 9ms/step - loss: 0.8895 - accuracy: 0.9049 - val_loss: 0.9265 - val_accuracy: 0.8993
Epoch 106/200
903/903 [==============================] - 8s 9ms/step - loss: 0.8972 - accuracy: 0.9053 - val_loss: 0.9216 - val_accuracy: 0.8980
Epoch 107/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8916 - accuracy: 0.9066 - val_loss: 0.9262 - val_accuracy: 0.8993
Epoch 108/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8961 - accuracy: 0.8991 - val_loss: 0.9108 - val_accuracy: 0.9013
Epoch 109/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8624 - accuracy: 0.9150 - val_loss: 0.9318 - val_accuracy: 0.8997
Epoch 110/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8787 - accuracy: 0.9068 - val_loss: 0.9424 - val_accuracy: 0.8910
Epoch 111/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8905 - accuracy: 0.9043 - val_loss: 0.9057 - val_accuracy: 0.9070
Epoch 112/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8900 - accuracy: 0.9046 - val_loss: 0.9207 - val_accuracy: 0.8987
Epoch 113/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8660 - accuracy: 0.9107 - val_loss: 0.8826 - val_accuracy: 0.9073
Epoch 114/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8740 - accuracy: 0.9114 - val_loss: 0.9023 - val_accuracy: 0.9013
Epoch 115/200
903/903 [==============================] - 7s 8ms/step - loss: 0.8665 - accuracy: 0.9132 - val_loss: 0.8699 - val_accuracy: 0.9143
Epoch 116/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8634 - accuracy: 0.9130 - val_loss: 0.8951 - val_accuracy: 0.9057
Epoch 117/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8583 - accuracy: 0.9115 - val_loss: 0.8867 - val_accuracy: 0.9087
Epoch 118/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8651 - accuracy: 0.9084 - val_loss: 0.8836 - val_accuracy: 0.9100
Epoch 119/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8605 - accuracy: 0.9126 - val_loss: 0.9222 - val_accuracy: 0.8953
Epoch 120/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8516 - accuracy: 0.9156 - val_loss: 0.8739 - val_accuracy: 0.9123
Epoch 121/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8855 - accuracy: 0.9067 - val_loss: 0.8901 - val_accuracy: 0.9093
Epoch 122/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8526 - accuracy: 0.9128 - val_loss: 0.8537 - val_accuracy: 0.9133
Epoch 123/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8532 - accuracy: 0.9150 - val_loss: 0.8829 - val_accuracy: 0.9070
Epoch 124/200
903/903 [==============================] - 7s 8ms/step - loss: 0.8265 - accuracy: 0.9188 - val_loss: 0.8725 - val_accuracy: 0.9083
Epoch 125/200
903/903 [==============================] - 8s 9ms/step - loss: 0.8568 - accuracy: 0.9124 - val_loss: 0.8851 - val_accuracy: 0.9123
Epoch 126/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8536 - accuracy: 0.9138 - val_loss: 0.8784 - val_accuracy: 0.9110
Epoch 127/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8443 - accuracy: 0.9136 - val_loss: 1.0094 - val_accuracy: 0.8767
Epoch 128/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8488 - accuracy: 0.9183 - val_loss: 0.8927 - val_accuracy: 0.9033
Epoch 129/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8262 - accuracy: 0.9232 - val_loss: 0.8929 - val_accuracy: 0.9047
Epoch 130/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8276 - accuracy: 0.9188 - val_loss: 0.8961 - val_accuracy: 0.9053
Epoch 131/200
903/903 [==============================] - 7s 8ms/step - loss: 0.8384 - accuracy: 0.9180 - val_loss: 0.8629 - val_accuracy: 0.9153
Epoch 132/200
903/903 [==============================] - 7s 8ms/step - loss: 0.8343 - accuracy: 0.9166 - val_loss: 0.8830 - val_accuracy: 0.9000
Epoch 133/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8284 - accuracy: 0.9208 - val_loss: 0.8757 - val_accuracy: 0.9087
Epoch 134/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8274 - accuracy: 0.9184 - val_loss: 0.8939 - val_accuracy: 0.9057
Epoch 135/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8394 - accuracy: 0.9177 - val_loss: 0.9388 - val_accuracy: 0.8960
Epoch 136/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8260 - accuracy: 0.9178 - val_loss: 0.8651 - val_accuracy: 0.9103
Epoch 137/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8345 - accuracy: 0.9202 - val_loss: 0.8560 - val_accuracy: 0.9147
Epoch 138/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8141 - accuracy: 0.9233 - val_loss: 0.8859 - val_accuracy: 0.9057
Epoch 139/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8106 - accuracy: 0.9209 - val_loss: 0.8589 - val_accuracy: 0.9160
Epoch 140/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8254 - accuracy: 0.9198 - val_loss: 0.8579 - val_accuracy: 0.9133
Epoch 141/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8081 - accuracy: 0.9233 - val_loss: 0.8769 - val_accuracy: 0.9067
Epoch 142/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8209 - accuracy: 0.9179 - val_loss: 0.8552 - val_accuracy: 0.9127
Epoch 143/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8028 - accuracy: 0.9253 - val_loss: 0.9285 - val_accuracy: 0.9010
Epoch 144/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8113 - accuracy: 0.9237 - val_loss: 0.8441 - val_accuracy: 0.9190
Epoch 145/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8007 - accuracy: 0.9252 - val_loss: 0.8496 - val_accuracy: 0.9173
Epoch 146/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8215 - accuracy: 0.9186 - val_loss: 0.8375 - val_accuracy: 0.9160
Epoch 147/200
903/903 [==============================] - 8s 9ms/step - loss: 0.7984 - accuracy: 0.9267 - val_loss: 0.8419 - val_accuracy: 0.9183
Epoch 148/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8197 - accuracy: 0.9191 - val_loss: 0.8460 - val_accuracy: 0.9167
Epoch 149/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8054 - accuracy: 0.9261 - val_loss: 0.8588 - val_accuracy: 0.9133
Epoch 150/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8007 - accuracy: 0.9245 - val_loss: 0.8517 - val_accuracy: 0.9133
Epoch 151/200
903/903 [==============================] - 7s 8ms/step - loss: 0.8110 - accuracy: 0.9202 - val_loss: 0.8753 - val_accuracy: 0.9063
Epoch 152/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8037 - accuracy: 0.9228 - val_loss: 0.8636 - val_accuracy: 0.9090
Epoch 153/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7955 - accuracy: 0.9274 - val_loss: 0.8450 - val_accuracy: 0.9150
Epoch 154/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7975 - accuracy: 0.9259 - val_loss: 0.8364 - val_accuracy: 0.9167
Epoch 155/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7937 - accuracy: 0.9263 - val_loss: 0.8344 - val_accuracy: 0.9200
Epoch 156/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8022 - accuracy: 0.9233 - val_loss: 0.8393 - val_accuracy: 0.9197
Epoch 157/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7960 - accuracy: 0.9267 - val_loss: 0.8784 - val_accuracy: 0.9080
Epoch 158/200
903/903 [==============================] - 8s 8ms/step - loss: 0.8078 - accuracy: 0.9212 - val_loss: 0.8438 - val_accuracy: 0.9197
Epoch 159/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7853 - accuracy: 0.9286 - val_loss: 0.8576 - val_accuracy: 0.9087
Epoch 160/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7948 - accuracy: 0.9246 - val_loss: 0.8194 - val_accuracy: 0.9247
Epoch 161/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7697 - accuracy: 0.9325 - val_loss: 0.8817 - val_accuracy: 0.9027
Epoch 162/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7932 - accuracy: 0.9282 - val_loss: 0.8231 - val_accuracy: 0.9173
Epoch 163/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7790 - accuracy: 0.9278 - val_loss: 0.8804 - val_accuracy: 0.9030
Epoch 164/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7882 - accuracy: 0.9239 - val_loss: 0.8630 - val_accuracy: 0.9037
Epoch 165/200
903/903 [==============================] - 7s 8ms/step - loss: 0.7921 - accuracy: 0.9260 - val_loss: 0.8681 - val_accuracy: 0.9070
Epoch 166/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7902 - accuracy: 0.9250 - val_loss: 0.8465 - val_accuracy: 0.9107
Epoch 167/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7754 - accuracy: 0.9290 - val_loss: 0.8508 - val_accuracy: 0.9113
Epoch 168/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7907 - accuracy: 0.9230 - val_loss: 0.8366 - val_accuracy: 0.9197
Epoch 169/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7825 - accuracy: 0.9299 - val_loss: 0.8480 - val_accuracy: 0.9140
Epoch 170/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7895 - accuracy: 0.9250 - val_loss: 0.8121 - val_accuracy: 0.9233
Epoch 171/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7634 - accuracy: 0.9304 - val_loss: 0.8716 - val_accuracy: 0.9080
Epoch 172/200
903/903 [==============================] - 7s 8ms/step - loss: 0.7736 - accuracy: 0.9307 - val_loss: 0.8377 - val_accuracy: 0.9130
Epoch 173/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7724 - accuracy: 0.9282 - val_loss: 0.8649 - val_accuracy: 0.9093
Epoch 174/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7806 - accuracy: 0.9277 - val_loss: 0.8493 - val_accuracy: 0.9150
Epoch 175/200
903/903 [==============================] - 7s 8ms/step - loss: 0.7951 - accuracy: 0.9235 - val_loss: 0.8223 - val_accuracy: 0.9167
Epoch 176/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7886 - accuracy: 0.9272 - val_loss: 0.8272 - val_accuracy: 0.9197
Epoch 177/200
903/903 [==============================] - 7s 8ms/step - loss: 0.7705 - accuracy: 0.9323 - val_loss: 0.8600 - val_accuracy: 0.9113
Epoch 178/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7716 - accuracy: 0.9286 - val_loss: 0.8246 - val_accuracy: 0.9247
Epoch 179/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7661 - accuracy: 0.9329 - val_loss: 0.8279 - val_accuracy: 0.9210
Epoch 180/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7595 - accuracy: 0.9331 - val_loss: 0.8657 - val_accuracy: 0.9090
Epoch 181/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7644 - accuracy: 0.9328 - val_loss: 0.7994 - val_accuracy: 0.9190
Epoch 182/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7552 - accuracy: 0.9342 - val_loss: 0.8147 - val_accuracy: 0.9200
Epoch 183/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7665 - accuracy: 0.9308 - val_loss: 0.8123 - val_accuracy: 0.9230
Epoch 184/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7595 - accuracy: 0.9314 - val_loss: 0.8468 - val_accuracy: 0.9147
Epoch 185/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7668 - accuracy: 0.9282 - val_loss: 0.8280 - val_accuracy: 0.9167
Epoch 186/200
903/903 [==============================] - 8s 9ms/step - loss: 0.7647 - accuracy: 0.9321 - val_loss: 0.8056 - val_accuracy: 0.9190
Epoch 187/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7545 - accuracy: 0.9338 - val_loss: 0.8183 - val_accuracy: 0.9170
Epoch 188/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7485 - accuracy: 0.9345 - val_loss: 0.8600 - val_accuracy: 0.9113
Epoch 189/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7668 - accuracy: 0.9294 - val_loss: 0.8133 - val_accuracy: 0.9217
Epoch 190/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7601 - accuracy: 0.9287 - val_loss: 0.8296 - val_accuracy: 0.9180
Epoch 191/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7507 - accuracy: 0.9355 - val_loss: 0.8104 - val_accuracy: 0.9230
Epoch 192/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7594 - accuracy: 0.9311 - val_loss: 0.8240 - val_accuracy: 0.9187
Epoch 193/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7572 - accuracy: 0.9333 - val_loss: 0.8244 - val_accuracy: 0.9213
Epoch 194/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7591 - accuracy: 0.9322 - val_loss: 0.8367 - val_accuracy: 0.9163
Epoch 195/200
903/903 [==============================] - 8s 9ms/step - loss: 0.7555 - accuracy: 0.9327 - val_loss: 0.8029 - val_accuracy: 0.9210
Epoch 196/200
903/903 [==============================] - 8s 9ms/step - loss: 0.7646 - accuracy: 0.9300 - val_loss: 0.8449 - val_accuracy: 0.9177
Epoch 197/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7482 - accuracy: 0.9333 - val_loss: 0.8377 - val_accuracy: 0.9173
Epoch 198/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7644 - accuracy: 0.9309 - val_loss: 0.8079 - val_accuracy: 0.9247
Epoch 199/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7388 - accuracy: 0.9369 - val_loss: 0.7893 - val_accuracy: 0.9307
Epoch 200/200
903/903 [==============================] - 8s 8ms/step - loss: 0.7471 - accuracy: 0.9369 - val_loss: 0.8143 - val_accuracy: 0.9197

plot_learning_curve(Conv2D_31_L2_history.history)

Observations

• The curves show how accurate the training and validation accuracy is after including L2, the losses experienced are also quite similar.

Conv2D_31_L2.evaluate(test_data_31.batch(10))

300/300 [==============================] - 2s 5ms/step - loss: 0.7772 - accuracy: 0.9347

[0.7772353291511536, 0.9346666932106018]

Observations

• Accuracy definitely improved after regularizing.

CNN Augmented Balance Regularisation Version 1

• I will attempt to improve the accuracy of the model to 93% through regularising

# Try for 31 x 31 images
tf.keras.backend.clear_session()

Conv2D_31_aug_l2 = Sequential(name="Conv2D_31_Augmentation_L2_Regularization",
    layers = [
        normalised_data,
        Conv2D(64, (5, 5),input_shape=(31, 31 ,1), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Conv2D(128, (3, 3), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Conv2D(256, (3, 3), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.5),

        Dense(256, activation='relu', kernel_regularizer=l2(0.01)),
        Dropout(0.5),
        BatchNormalization(),

        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.0001)
Conv2D_31_aug_l2.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_31_aug_l2.build(input_shape=(None, 31, 31, 1))

Conv2D_31_aug_l2_history = Conv2D_31_aug_l2.fit(
    train_31V2.batch(32), 
    epochs=200,
    validation_data=val_data_31.batch(32)
    )

Epoch 1/200
1433/1433 [==============================] - 6s 4ms/step - loss: 4.8458 - accuracy: 0.1430 - val_loss: 3.6078 - val_accuracy: 0.2090
Epoch 2/200
1433/1433 [==============================] - 6s 4ms/step - loss: 2.9239 - accuracy: 0.3238 - val_loss: 2.4861 - val_accuracy: 0.3667
Epoch 3/200
1433/1433 [==============================] - 6s 4ms/step - loss: 2.1846 - accuracy: 0.4416 - val_loss: 2.1041 - val_accuracy: 0.4450
Epoch 4/200
1433/1433 [==============================] - 6s 4ms/step - loss: 1.8330 - accuracy: 0.5298 - val_loss: 1.8714 - val_accuracy: 0.4960
Epoch 5/200
1433/1433 [==============================] - 6s 4ms/step - loss: 1.5933 - accuracy: 0.5932 - val_loss: 1.6311 - val_accuracy: 0.5703
Epoch 6/200
1433/1433 [==============================] - 6s 4ms/step - loss: 1.4237 - accuracy: 0.6401 - val_loss: 1.4915 - val_accuracy: 0.6070
Epoch 7/200
1433/1433 [==============================] - 6s 4ms/step - loss: 1.2976 - accuracy: 0.6856 - val_loss: 1.4689 - val_accuracy: 0.6220
Epoch 8/200
1433/1433 [==============================] - 5s 4ms/step - loss: 1.1892 - accuracy: 0.7188 - val_loss: 1.3708 - val_accuracy: 0.6527
Epoch 9/200
1433/1433 [==============================] - 5s 4ms/step - loss: 1.1272 - accuracy: 0.7373 - val_loss: 1.0786 - val_accuracy: 0.7400
Epoch 10/200
1433/1433 [==============================] - 5s 4ms/step - loss: 1.0579 - accuracy: 0.7588 - val_loss: 0.9798 - val_accuracy: 0.7763
Epoch 11/200
1433/1433 [==============================] - 5s 4ms/step - loss: 1.0043 - accuracy: 0.7755 - val_loss: 1.2971 - val_accuracy: 0.6750
Epoch 12/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.9510 - accuracy: 0.7926 - val_loss: 1.1713 - val_accuracy: 0.7133
Epoch 13/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.9178 - accuracy: 0.8018 - val_loss: 1.2294 - val_accuracy: 0.6987
Epoch 14/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.8738 - accuracy: 0.8138 - val_loss: 1.1532 - val_accuracy: 0.7237
Epoch 15/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.8425 - accuracy: 0.8261 - val_loss: 1.0566 - val_accuracy: 0.7523
Epoch 16/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.8094 - accuracy: 0.8336 - val_loss: 0.9408 - val_accuracy: 0.7933
Epoch 17/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.7779 - accuracy: 0.8427 - val_loss: 1.0536 - val_accuracy: 0.7687
Epoch 18/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.7555 - accuracy: 0.8508 - val_loss: 0.9835 - val_accuracy: 0.7837
Epoch 19/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.7312 - accuracy: 0.8602 - val_loss: 0.9887 - val_accuracy: 0.7737
Epoch 20/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.7134 - accuracy: 0.8634 - val_loss: 1.0615 - val_accuracy: 0.7630
Epoch 21/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.7000 - accuracy: 0.8679 - val_loss: 0.7791 - val_accuracy: 0.8470
Epoch 22/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.6761 - accuracy: 0.8743 - val_loss: 0.9162 - val_accuracy: 0.8090
Epoch 23/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.6574 - accuracy: 0.8792 - val_loss: 0.9339 - val_accuracy: 0.8017
Epoch 24/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.6522 - accuracy: 0.8813 - val_loss: 0.7555 - val_accuracy: 0.8500
Epoch 25/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.6295 - accuracy: 0.8869 - val_loss: 0.7046 - val_accuracy: 0.8753
Epoch 26/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.6231 - accuracy: 0.8906 - val_loss: 0.8411 - val_accuracy: 0.8220
Epoch 27/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.6071 - accuracy: 0.8942 - val_loss: 0.7459 - val_accuracy: 0.8577
Epoch 28/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.5937 - accuracy: 0.8983 - val_loss: 0.7407 - val_accuracy: 0.8663
Epoch 29/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5930 - accuracy: 0.8972 - val_loss: 0.8478 - val_accuracy: 0.8363
Epoch 30/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5888 - accuracy: 0.9012 - val_loss: 0.6476 - val_accuracy: 0.8880
Epoch 31/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5631 - accuracy: 0.9090 - val_loss: 0.8860 - val_accuracy: 0.8233
Epoch 32/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5630 - accuracy: 0.9084 - val_loss: 0.7892 - val_accuracy: 0.8470
Epoch 33/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5410 - accuracy: 0.9162 - val_loss: 0.7154 - val_accuracy: 0.8670
Epoch 34/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5465 - accuracy: 0.9109 - val_loss: 0.7222 - val_accuracy: 0.8663
Epoch 35/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5403 - accuracy: 0.9127 - val_loss: 0.8873 - val_accuracy: 0.8203
Epoch 36/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5142 - accuracy: 0.9221 - val_loss: 0.8357 - val_accuracy: 0.8383
Epoch 37/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5198 - accuracy: 0.9178 - val_loss: 0.8731 - val_accuracy: 0.8220
Epoch 38/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5094 - accuracy: 0.9249 - val_loss: 0.9974 - val_accuracy: 0.8007
Epoch 39/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.5128 - accuracy: 0.9192 - val_loss: 0.8602 - val_accuracy: 0.8333
Epoch 40/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4993 - accuracy: 0.9270 - val_loss: 0.6696 - val_accuracy: 0.8840
Epoch 41/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4997 - accuracy: 0.9259 - val_loss: 1.3629 - val_accuracy: 0.7260
Epoch 42/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4845 - accuracy: 0.9282 - val_loss: 0.7632 - val_accuracy: 0.8630
Epoch 43/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4810 - accuracy: 0.9285 - val_loss: 0.7604 - val_accuracy: 0.8623
Epoch 44/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4737 - accuracy: 0.9331 - val_loss: 0.6341 - val_accuracy: 0.8967
Epoch 45/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4735 - accuracy: 0.9319 - val_loss: 0.7305 - val_accuracy: 0.8687
Epoch 46/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4646 - accuracy: 0.9342 - val_loss: 0.6497 - val_accuracy: 0.8933
Epoch 47/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4546 - accuracy: 0.9341 - val_loss: 0.7784 - val_accuracy: 0.8487
Epoch 48/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4591 - accuracy: 0.9338 - val_loss: 0.8860 - val_accuracy: 0.8233
Epoch 49/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4498 - accuracy: 0.9351 - val_loss: 0.9472 - val_accuracy: 0.8150
Epoch 50/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.4416 - accuracy: 0.9379 - val_loss: 0.9503 - val_accuracy: 0.8170
Epoch 51/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.4485 - accuracy: 0.9372 - val_loss: 0.8730 - val_accuracy: 0.8360
Epoch 52/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4502 - accuracy: 0.9329 - val_loss: 0.7651 - val_accuracy: 0.8683
Epoch 53/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.4263 - accuracy: 0.9436 - val_loss: 0.6665 - val_accuracy: 0.8893
Epoch 54/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4334 - accuracy: 0.9414 - val_loss: 0.8806 - val_accuracy: 0.8373
Epoch 55/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.4248 - accuracy: 0.9435 - val_loss: 0.7849 - val_accuracy: 0.8577
Epoch 56/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.4250 - accuracy: 0.9443 - val_loss: 0.7057 - val_accuracy: 0.8813
Epoch 57/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4261 - accuracy: 0.9404 - val_loss: 0.8515 - val_accuracy: 0.8387
Epoch 58/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4182 - accuracy: 0.9449 - val_loss: 0.6042 - val_accuracy: 0.9040
Epoch 59/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4021 - accuracy: 0.9499 - val_loss: 0.5714 - val_accuracy: 0.9133
Epoch 60/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4037 - accuracy: 0.9485 - val_loss: 0.9108 - val_accuracy: 0.8380
Epoch 61/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4098 - accuracy: 0.9444 - val_loss: 0.7955 - val_accuracy: 0.8607
Epoch 62/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4024 - accuracy: 0.9488 - val_loss: 0.6925 - val_accuracy: 0.8853
Epoch 63/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.4033 - accuracy: 0.9491 - val_loss: 0.6957 - val_accuracy: 0.8883
Epoch 64/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3976 - accuracy: 0.9488 - val_loss: 1.0236 - val_accuracy: 0.8143
Epoch 65/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3931 - accuracy: 0.9514 - val_loss: 0.6493 - val_accuracy: 0.8940
Epoch 66/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3907 - accuracy: 0.9514 - val_loss: 0.6073 - val_accuracy: 0.9087
Epoch 67/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3883 - accuracy: 0.9519 - val_loss: 0.6395 - val_accuracy: 0.8983
Epoch 68/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3900 - accuracy: 0.9513 - val_loss: 0.8671 - val_accuracy: 0.8397
Epoch 69/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3872 - accuracy: 0.9505 - val_loss: 0.7011 - val_accuracy: 0.8797
Epoch 70/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3765 - accuracy: 0.9540 - val_loss: 0.6070 - val_accuracy: 0.9073
Epoch 71/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3808 - accuracy: 0.9529 - val_loss: 0.7971 - val_accuracy: 0.8667
Epoch 72/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3792 - accuracy: 0.9532 - val_loss: 0.8593 - val_accuracy: 0.8520
Epoch 73/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3737 - accuracy: 0.9544 - val_loss: 0.6351 - val_accuracy: 0.8977
Epoch 74/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3773 - accuracy: 0.9543 - val_loss: 0.7259 - val_accuracy: 0.8750
Epoch 75/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3676 - accuracy: 0.9552 - val_loss: 0.6924 - val_accuracy: 0.8850
Epoch 76/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3620 - accuracy: 0.9587 - val_loss: 0.7021 - val_accuracy: 0.8827
Epoch 77/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3616 - accuracy: 0.9573 - val_loss: 0.8194 - val_accuracy: 0.8580
Epoch 78/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3679 - accuracy: 0.9543 - val_loss: 0.7283 - val_accuracy: 0.8783
Epoch 79/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3614 - accuracy: 0.9557 - val_loss: 0.9172 - val_accuracy: 0.8350
Epoch 80/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3551 - accuracy: 0.9576 - val_loss: 1.0690 - val_accuracy: 0.7987
Epoch 81/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3642 - accuracy: 0.9545 - val_loss: 0.8052 - val_accuracy: 0.8527
Epoch 82/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3655 - accuracy: 0.9566 - val_loss: 0.7031 - val_accuracy: 0.8813
Epoch 83/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3595 - accuracy: 0.9548 - val_loss: 0.8217 - val_accuracy: 0.8603
Epoch 84/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3552 - accuracy: 0.9597 - val_loss: 0.8216 - val_accuracy: 0.8563
Epoch 85/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3489 - accuracy: 0.9597 - val_loss: 0.6956 - val_accuracy: 0.8763
Epoch 86/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3472 - accuracy: 0.9599 - val_loss: 0.7004 - val_accuracy: 0.8817
Epoch 87/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3489 - accuracy: 0.9597 - val_loss: 0.8485 - val_accuracy: 0.8467
Epoch 88/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3405 - accuracy: 0.9616 - val_loss: 0.6298 - val_accuracy: 0.9080
Epoch 89/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3478 - accuracy: 0.9608 - val_loss: 0.9921 - val_accuracy: 0.8157
Epoch 90/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3447 - accuracy: 0.9606 - val_loss: 0.9137 - val_accuracy: 0.8353
Epoch 91/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3541 - accuracy: 0.9562 - val_loss: 0.6759 - val_accuracy: 0.8893
Epoch 92/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3400 - accuracy: 0.9610 - val_loss: 0.8524 - val_accuracy: 0.8517
Epoch 93/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3393 - accuracy: 0.9628 - val_loss: 0.8096 - val_accuracy: 0.8607
Epoch 94/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3401 - accuracy: 0.9603 - val_loss: 0.7690 - val_accuracy: 0.8703
Epoch 95/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3423 - accuracy: 0.9603 - val_loss: 0.9463 - val_accuracy: 0.8363
Epoch 96/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3266 - accuracy: 0.9658 - val_loss: 0.6798 - val_accuracy: 0.8903
Epoch 97/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3265 - accuracy: 0.9633 - val_loss: 0.6255 - val_accuracy: 0.9040
Epoch 98/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3335 - accuracy: 0.9622 - val_loss: 0.7855 - val_accuracy: 0.8707
Epoch 99/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3200 - accuracy: 0.9653 - val_loss: 0.7896 - val_accuracy: 0.8737
Epoch 100/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3283 - accuracy: 0.9642 - val_loss: 0.6288 - val_accuracy: 0.9047
Epoch 101/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3194 - accuracy: 0.9669 - val_loss: 0.8715 - val_accuracy: 0.8607
Epoch 102/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3231 - accuracy: 0.9644 - val_loss: 0.7347 - val_accuracy: 0.8810
Epoch 103/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3224 - accuracy: 0.9647 - val_loss: 0.6367 - val_accuracy: 0.9020
Epoch 104/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3251 - accuracy: 0.9650 - val_loss: 0.9211 - val_accuracy: 0.8443
Epoch 105/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3323 - accuracy: 0.9620 - val_loss: 1.1170 - val_accuracy: 0.8003
Epoch 106/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3188 - accuracy: 0.9664 - val_loss: 0.6492 - val_accuracy: 0.9010
Epoch 107/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3221 - accuracy: 0.9649 - val_loss: 0.8842 - val_accuracy: 0.8620
Epoch 108/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3210 - accuracy: 0.9651 - val_loss: 0.7209 - val_accuracy: 0.8820
Epoch 109/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3189 - accuracy: 0.9665 - val_loss: 0.6916 - val_accuracy: 0.8877
Epoch 110/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3147 - accuracy: 0.9673 - val_loss: 0.7489 - val_accuracy: 0.8720
Epoch 111/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3146 - accuracy: 0.9664 - val_loss: 0.7637 - val_accuracy: 0.8733
Epoch 112/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3173 - accuracy: 0.9646 - val_loss: 0.6664 - val_accuracy: 0.8940
Epoch 113/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3097 - accuracy: 0.9683 - val_loss: 0.7000 - val_accuracy: 0.8843
Epoch 114/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3085 - accuracy: 0.9709 - val_loss: 0.7687 - val_accuracy: 0.8827
Epoch 115/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3143 - accuracy: 0.9650 - val_loss: 0.8392 - val_accuracy: 0.8620
Epoch 116/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3066 - accuracy: 0.9672 - val_loss: 0.7224 - val_accuracy: 0.8840
Epoch 117/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3024 - accuracy: 0.9694 - val_loss: 0.8523 - val_accuracy: 0.8593
Epoch 118/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3129 - accuracy: 0.9668 - val_loss: 0.7045 - val_accuracy: 0.8853
Epoch 119/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2997 - accuracy: 0.9695 - val_loss: 1.2329 - val_accuracy: 0.8017
Epoch 120/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.3106 - accuracy: 0.9665 - val_loss: 0.5985 - val_accuracy: 0.9130
Epoch 121/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3147 - accuracy: 0.9669 - val_loss: 0.6105 - val_accuracy: 0.9073
Epoch 122/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3059 - accuracy: 0.9690 - val_loss: 0.7698 - val_accuracy: 0.8713
Epoch 123/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3050 - accuracy: 0.9687 - val_loss: 0.8783 - val_accuracy: 0.8637
Epoch 124/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3046 - accuracy: 0.9694 - val_loss: 0.9405 - val_accuracy: 0.8470
Epoch 125/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2970 - accuracy: 0.9714 - val_loss: 0.8370 - val_accuracy: 0.8590
Epoch 126/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2925 - accuracy: 0.9714 - val_loss: 0.8743 - val_accuracy: 0.8557
Epoch 127/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2957 - accuracy: 0.9698 - val_loss: 0.7954 - val_accuracy: 0.8707
Epoch 128/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3076 - accuracy: 0.9659 - val_loss: 0.8928 - val_accuracy: 0.8603
Epoch 129/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.3037 - accuracy: 0.9677 - val_loss: 0.6578 - val_accuracy: 0.9017
Epoch 130/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2992 - accuracy: 0.9697 - val_loss: 0.6810 - val_accuracy: 0.8903
Epoch 131/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2901 - accuracy: 0.9728 - val_loss: 0.6937 - val_accuracy: 0.8910
Epoch 132/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2936 - accuracy: 0.9715 - val_loss: 0.6462 - val_accuracy: 0.8920
Epoch 133/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2973 - accuracy: 0.9696 - val_loss: 0.7298 - val_accuracy: 0.8823
Epoch 134/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2981 - accuracy: 0.9695 - val_loss: 0.6352 - val_accuracy: 0.8993
Epoch 135/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2886 - accuracy: 0.9705 - val_loss: 0.7712 - val_accuracy: 0.8753
Epoch 136/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2941 - accuracy: 0.9705 - val_loss: 0.7428 - val_accuracy: 0.8813
Epoch 137/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2900 - accuracy: 0.9714 - val_loss: 1.1047 - val_accuracy: 0.8057
Epoch 138/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2868 - accuracy: 0.9717 - val_loss: 0.7429 - val_accuracy: 0.8850
Epoch 139/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2897 - accuracy: 0.9707 - val_loss: 0.7243 - val_accuracy: 0.8873
Epoch 140/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2960 - accuracy: 0.9698 - val_loss: 0.8444 - val_accuracy: 0.8650
Epoch 141/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2907 - accuracy: 0.9699 - val_loss: 0.9882 - val_accuracy: 0.8440
Epoch 142/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2851 - accuracy: 0.9732 - val_loss: 1.0721 - val_accuracy: 0.8207
Epoch 143/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2786 - accuracy: 0.9728 - val_loss: 0.6574 - val_accuracy: 0.9023
Epoch 144/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2763 - accuracy: 0.9736 - val_loss: 0.7528 - val_accuracy: 0.8780
Epoch 145/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2880 - accuracy: 0.9702 - val_loss: 0.7042 - val_accuracy: 0.8853
Epoch 146/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2885 - accuracy: 0.9701 - val_loss: 0.5786 - val_accuracy: 0.9117
Epoch 147/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2893 - accuracy: 0.9710 - val_loss: 0.9758 - val_accuracy: 0.8413
Epoch 148/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2787 - accuracy: 0.9736 - val_loss: 0.8386 - val_accuracy: 0.8667
Epoch 149/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2772 - accuracy: 0.9738 - val_loss: 0.9030 - val_accuracy: 0.8570
Epoch 150/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2862 - accuracy: 0.9712 - val_loss: 0.8266 - val_accuracy: 0.8727
Epoch 151/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2859 - accuracy: 0.9708 - val_loss: 0.7918 - val_accuracy: 0.8820
Epoch 152/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2841 - accuracy: 0.9714 - val_loss: 0.8495 - val_accuracy: 0.8640
Epoch 153/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2801 - accuracy: 0.9748 - val_loss: 0.7721 - val_accuracy: 0.8793
Epoch 154/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2801 - accuracy: 0.9721 - val_loss: 0.7301 - val_accuracy: 0.8920
Epoch 155/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2792 - accuracy: 0.9721 - val_loss: 0.7533 - val_accuracy: 0.8800
Epoch 156/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2833 - accuracy: 0.9715 - val_loss: 0.9055 - val_accuracy: 0.8467
Epoch 157/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2807 - accuracy: 0.9727 - val_loss: 0.7244 - val_accuracy: 0.8837
Epoch 158/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2731 - accuracy: 0.9742 - val_loss: 0.8174 - val_accuracy: 0.8657
Epoch 159/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2670 - accuracy: 0.9743 - val_loss: 0.7136 - val_accuracy: 0.8883
Epoch 160/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2781 - accuracy: 0.9733 - val_loss: 0.7281 - val_accuracy: 0.8843
Epoch 161/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2683 - accuracy: 0.9756 - val_loss: 0.8218 - val_accuracy: 0.8680
Epoch 162/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2715 - accuracy: 0.9732 - val_loss: 0.6900 - val_accuracy: 0.8910
Epoch 163/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2707 - accuracy: 0.9740 - val_loss: 0.6357 - val_accuracy: 0.9017
Epoch 164/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2716 - accuracy: 0.9751 - val_loss: 0.6945 - val_accuracy: 0.8963
Epoch 165/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2755 - accuracy: 0.9723 - val_loss: 0.6133 - val_accuracy: 0.9057
Epoch 166/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2674 - accuracy: 0.9758 - val_loss: 0.7635 - val_accuracy: 0.8817
Epoch 167/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2702 - accuracy: 0.9731 - val_loss: 0.7991 - val_accuracy: 0.8760
Epoch 168/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2717 - accuracy: 0.9743 - val_loss: 0.6843 - val_accuracy: 0.8923
Epoch 169/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2703 - accuracy: 0.9749 - val_loss: 1.0022 - val_accuracy: 0.8283
Epoch 170/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2682 - accuracy: 0.9743 - val_loss: 0.6885 - val_accuracy: 0.8960
Epoch 171/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2719 - accuracy: 0.9736 - val_loss: 0.7024 - val_accuracy: 0.8970
Epoch 172/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2707 - accuracy: 0.9742 - val_loss: 0.6104 - val_accuracy: 0.9110
Epoch 173/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2805 - accuracy: 0.9719 - val_loss: 0.6906 - val_accuracy: 0.8977
Epoch 174/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2624 - accuracy: 0.9772 - val_loss: 0.7732 - val_accuracy: 0.8817
Epoch 175/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2596 - accuracy: 0.9765 - val_loss: 0.7564 - val_accuracy: 0.8847
Epoch 176/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2623 - accuracy: 0.9756 - val_loss: 1.0207 - val_accuracy: 0.8343
Epoch 177/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2689 - accuracy: 0.9747 - val_loss: 0.6977 - val_accuracy: 0.8997
Epoch 178/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2638 - accuracy: 0.9742 - val_loss: 0.8574 - val_accuracy: 0.8643
Epoch 179/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2758 - accuracy: 0.9730 - val_loss: 0.8502 - val_accuracy: 0.8650
Epoch 180/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2665 - accuracy: 0.9762 - val_loss: 0.6111 - val_accuracy: 0.9010
Epoch 181/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2644 - accuracy: 0.9756 - val_loss: 0.6527 - val_accuracy: 0.8870
Epoch 182/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2657 - accuracy: 0.9741 - val_loss: 1.1372 - val_accuracy: 0.8193
Epoch 183/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2578 - accuracy: 0.9771 - val_loss: 0.5697 - val_accuracy: 0.9170
Epoch 184/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2686 - accuracy: 0.9744 - val_loss: 0.7753 - val_accuracy: 0.8817
Epoch 185/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2688 - accuracy: 0.9752 - val_loss: 0.7401 - val_accuracy: 0.8890
Epoch 186/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2574 - accuracy: 0.9768 - val_loss: 1.0511 - val_accuracy: 0.8307
Epoch 187/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2635 - accuracy: 0.9749 - val_loss: 0.9142 - val_accuracy: 0.8613
Epoch 188/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2657 - accuracy: 0.9755 - val_loss: 0.6814 - val_accuracy: 0.8940
Epoch 189/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2599 - accuracy: 0.9767 - val_loss: 0.6905 - val_accuracy: 0.8990
Epoch 190/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2603 - accuracy: 0.9763 - val_loss: 0.6408 - val_accuracy: 0.8983
Epoch 191/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2613 - accuracy: 0.9761 - val_loss: 0.7678 - val_accuracy: 0.8797
Epoch 192/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2608 - accuracy: 0.9772 - val_loss: 0.7186 - val_accuracy: 0.8850
Epoch 193/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2568 - accuracy: 0.9757 - val_loss: 0.7007 - val_accuracy: 0.8920
Epoch 194/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2590 - accuracy: 0.9761 - val_loss: 0.7740 - val_accuracy: 0.8843
Epoch 195/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2565 - accuracy: 0.9767 - val_loss: 0.7678 - val_accuracy: 0.8780
Epoch 196/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2598 - accuracy: 0.9740 - val_loss: 0.7819 - val_accuracy: 0.8760
Epoch 197/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2405 - accuracy: 0.9805 - val_loss: 0.8100 - val_accuracy: 0.8760
Epoch 198/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2502 - accuracy: 0.9768 - val_loss: 0.7212 - val_accuracy: 0.8880
Epoch 199/200
1433/1433 [==============================] - 5s 4ms/step - loss: 0.2550 - accuracy: 0.9754 - val_loss: 0.5745 - val_accuracy: 0.9117
Epoch 200/200
1433/1433 [==============================] - 6s 4ms/step - loss: 0.2583 - accuracy: 0.9768 - val_loss: 0.5944 - val_accuracy: 0.9137

plot_learning_curve(Conv2D_31_aug_l2_history.history)

Observations

• From what I can see, the validation loss is still higher than the training loss, unable to converge as well as the unaugmented data.
• For accuracy, the validation accuracy is still lower than the training accuracy.

Conv2D_31_aug_l2.evaluate(test_data_31.batch(10))

300/300 [==============================] - 1s 3ms/step - loss: 0.6360 - accuracy: 0.9050

[0.6360276341438293, 0.9049999713897705]

Observations

• The test accuracy may be quite high, reaching 90% accuracy but it was deemed lucky as the learning curve has shown.

CNN Augmented Balance Regularisation Version 2

• Since batch normalisation is used, I will attempt to increase the batch size as batch normalisation tends to work better for data with bigger batch size.

# Try for 31 x 31 images
tf.keras.backend.clear_session()

Conv2D_31_aug_l2_V2 = Sequential(name="Conv2D_31_Augmentation_L2_Regularization_Version2",
    layers = [
        normalised_data,
        Conv2D(64, (5, 5),input_shape=(31, 31 ,1), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Conv2D(128, (3, 3), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Conv2D(256, (3, 3), activation='relu', padding='same'),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dropout(0.6),

        Dense(256, activation='relu', kernel_regularizer=l2(0.01)),
        Dropout(0.6),
        BatchNormalization(),

        Dense(num_classes, activation='softmax')
    ]
)

# Compile model
opt = Adam(learning_rate=0.0001)
Conv2D_31_aug_l2_V2.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Conv2D_31_aug_l2_V2.build(input_shape=(None, 31, 31, 1))

Conv2D_31_aug_l2_V2_history = Conv2D_31_aug_l2_V2.fit(
    train_31V2.batch(32), 
    epochs=200,
    validation_data=val_data_31.batch(32)
    )

Epoch 1/200
448/448 [==============================] - 3s 5ms/step - loss: 5.8983 - accuracy: 0.1011 - val_loss: 4.8795 - val_accuracy: 0.2317
Epoch 2/200
448/448 [==============================] - 2s 4ms/step - loss: 4.1911 - accuracy: 0.2115 - val_loss: 3.6575 - val_accuracy: 0.2633
Epoch 3/200
448/448 [==============================] - 2s 4ms/step - loss: 3.2832 - accuracy: 0.3083 - val_loss: 3.0719 - val_accuracy: 0.3100
Epoch 4/200
448/448 [==============================] - 2s 4ms/step - loss: 2.7060 - accuracy: 0.3895 - val_loss: 2.3375 - val_accuracy: 0.4817
Epoch 5/200
448/448 [==============================] - 2s 4ms/step - loss: 2.3172 - accuracy: 0.4508 - val_loss: 2.3120 - val_accuracy: 0.4207
Epoch 6/200
448/448 [==============================] - 2s 4ms/step - loss: 2.0300 - accuracy: 0.5076 - val_loss: 1.9782 - val_accuracy: 0.5087
Epoch 7/200
448/448 [==============================] - 2s 4ms/step - loss: 1.8234 - accuracy: 0.5559 - val_loss: 1.6882 - val_accuracy: 0.5853
Epoch 8/200
448/448 [==============================] - 2s 4ms/step - loss: 1.6569 - accuracy: 0.5942 - val_loss: 1.6824 - val_accuracy: 0.5753
Epoch 9/200
448/448 [==============================] - 2s 4ms/step - loss: 1.5364 - accuracy: 0.6165 - val_loss: 1.5109 - val_accuracy: 0.6227
Epoch 10/200
448/448 [==============================] - 2s 4ms/step - loss: 1.4245 - accuracy: 0.6533 - val_loss: 1.6119 - val_accuracy: 0.5740
Epoch 11/200
448/448 [==============================] - 2s 4ms/step - loss: 1.3403 - accuracy: 0.6720 - val_loss: 1.6568 - val_accuracy: 0.5513
Epoch 12/200
448/448 [==============================] - 2s 4ms/step - loss: 1.2540 - accuracy: 0.6991 - val_loss: 1.3677 - val_accuracy: 0.6547
Epoch 13/200
448/448 [==============================] - 2s 4ms/step - loss: 1.1872 - accuracy: 0.7201 - val_loss: 1.2363 - val_accuracy: 0.6870
Epoch 14/200
448/448 [==============================] - 2s 4ms/step - loss: 1.1336 - accuracy: 0.7328 - val_loss: 1.1279 - val_accuracy: 0.7263
Epoch 15/200
448/448 [==============================] - 2s 4ms/step - loss: 1.0752 - accuracy: 0.7506 - val_loss: 1.1735 - val_accuracy: 0.7090
Epoch 16/200
448/448 [==============================] - 2s 4ms/step - loss: 1.0356 - accuracy: 0.7602 - val_loss: 1.0338 - val_accuracy: 0.7610
Epoch 17/200
448/448 [==============================] - 2s 4ms/step - loss: 0.9911 - accuracy: 0.7721 - val_loss: 1.0617 - val_accuracy: 0.7420
Epoch 18/200
448/448 [==============================] - 2s 4ms/step - loss: 0.9535 - accuracy: 0.7837 - val_loss: 0.9863 - val_accuracy: 0.7683
Epoch 19/200
448/448 [==============================] - 2s 4ms/step - loss: 0.9182 - accuracy: 0.7936 - val_loss: 1.0696 - val_accuracy: 0.7403
Epoch 20/200
448/448 [==============================] - 2s 4ms/step - loss: 0.8915 - accuracy: 0.8046 - val_loss: 0.9990 - val_accuracy: 0.7620
Epoch 21/200
448/448 [==============================] - 2s 4ms/step - loss: 0.8797 - accuracy: 0.8011 - val_loss: 0.9023 - val_accuracy: 0.7977
Epoch 22/200
448/448 [==============================] - 2s 4ms/step - loss: 0.8316 - accuracy: 0.8164 - val_loss: 0.9418 - val_accuracy: 0.7720
Epoch 23/200
448/448 [==============================] - 2s 4ms/step - loss: 0.8241 - accuracy: 0.8168 - val_loss: 0.8919 - val_accuracy: 0.7940
Epoch 24/200
448/448 [==============================] - 2s 4ms/step - loss: 0.8032 - accuracy: 0.8244 - val_loss: 0.8108 - val_accuracy: 0.8243
Epoch 25/200
448/448 [==============================] - 2s 4ms/step - loss: 0.7790 - accuracy: 0.8318 - val_loss: 0.8639 - val_accuracy: 0.8077
Epoch 26/200
448/448 [==============================] - 2s 4ms/step - loss: 0.7646 - accuracy: 0.8378 - val_loss: 1.0346 - val_accuracy: 0.7460
Epoch 27/200
448/448 [==============================] - 2s 4ms/step - loss: 0.7374 - accuracy: 0.8455 - val_loss: 0.8178 - val_accuracy: 0.8177
Epoch 28/200
448/448 [==============================] - 2s 4ms/step - loss: 0.7241 - accuracy: 0.8482 - val_loss: 0.8907 - val_accuracy: 0.7980
Epoch 29/200
448/448 [==============================] - 2s 4ms/step - loss: 0.7141 - accuracy: 0.8544 - val_loss: 0.8239 - val_accuracy: 0.8213
Epoch 30/200
448/448 [==============================] - 2s 4ms/step - loss: 0.7096 - accuracy: 0.8527 - val_loss: 0.8149 - val_accuracy: 0.8250
Epoch 31/200
448/448 [==============================] - 2s 4ms/step - loss: 0.6882 - accuracy: 0.8584 - val_loss: 0.8392 - val_accuracy: 0.8120
Epoch 32/200
448/448 [==============================] - 2s 4ms/step - loss: 0.6663 - accuracy: 0.8667 - val_loss: 0.7655 - val_accuracy: 0.8340
Epoch 33/200
448/448 [==============================] - 2s 4ms/step - loss: 0.6556 - accuracy: 0.8697 - val_loss: 0.9913 - val_accuracy: 0.7660
Epoch 34/200
448/448 [==============================] - 2s 4ms/step - loss: 0.6478 - accuracy: 0.8689 - val_loss: 0.8667 - val_accuracy: 0.8010
Epoch 35/200
448/448 [==============================] - 2s 4ms/step - loss: 0.6415 - accuracy: 0.8718 - val_loss: 0.9083 - val_accuracy: 0.7937
Epoch 36/200
448/448 [==============================] - 2s 4ms/step - loss: 0.6259 - accuracy: 0.8764 - val_loss: 0.8532 - val_accuracy: 0.8050
Epoch 37/200
448/448 [==============================] - 2s 4ms/step - loss: 0.6171 - accuracy: 0.8772 - val_loss: 0.8185 - val_accuracy: 0.8217
Epoch 38/200
448/448 [==============================] - 2s 4ms/step - loss: 0.6153 - accuracy: 0.8815 - val_loss: 0.8325 - val_accuracy: 0.8133
Epoch 39/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5937 - accuracy: 0.8871 - val_loss: 0.7195 - val_accuracy: 0.8527
Epoch 40/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5965 - accuracy: 0.8842 - val_loss: 0.8075 - val_accuracy: 0.8260
Epoch 41/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5846 - accuracy: 0.8903 - val_loss: 0.7592 - val_accuracy: 0.8393
Epoch 42/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5745 - accuracy: 0.8913 - val_loss: 0.7586 - val_accuracy: 0.8357
Epoch 43/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5683 - accuracy: 0.8931 - val_loss: 0.8282 - val_accuracy: 0.8133
Epoch 44/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5643 - accuracy: 0.8968 - val_loss: 0.7091 - val_accuracy: 0.8547
Epoch 45/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5499 - accuracy: 0.8979 - val_loss: 0.7243 - val_accuracy: 0.8483
Epoch 46/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5472 - accuracy: 0.8990 - val_loss: 0.8917 - val_accuracy: 0.7980
Epoch 47/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5351 - accuracy: 0.9030 - val_loss: 0.6908 - val_accuracy: 0.8583
Epoch 48/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5316 - accuracy: 0.9024 - val_loss: 0.9175 - val_accuracy: 0.7860
Epoch 49/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5333 - accuracy: 0.9025 - val_loss: 0.7305 - val_accuracy: 0.8533
Epoch 50/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5159 - accuracy: 0.9079 - val_loss: 0.7072 - val_accuracy: 0.8507
Epoch 51/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5235 - accuracy: 0.9049 - val_loss: 0.8136 - val_accuracy: 0.8270
Epoch 52/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5152 - accuracy: 0.9076 - val_loss: 0.7241 - val_accuracy: 0.8470
Epoch 53/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5134 - accuracy: 0.9074 - val_loss: 0.6529 - val_accuracy: 0.8690
Epoch 54/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5068 - accuracy: 0.9116 - val_loss: 0.6972 - val_accuracy: 0.8530
Epoch 55/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5006 - accuracy: 0.9131 - val_loss: 0.7652 - val_accuracy: 0.8327
Epoch 56/200
448/448 [==============================] - 2s 4ms/step - loss: 0.5003 - accuracy: 0.9092 - val_loss: 0.9235 - val_accuracy: 0.7877
Epoch 57/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4804 - accuracy: 0.9199 - val_loss: 0.7225 - val_accuracy: 0.8503
Epoch 58/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4835 - accuracy: 0.9199 - val_loss: 0.6600 - val_accuracy: 0.8670
Epoch 59/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4803 - accuracy: 0.9159 - val_loss: 0.6743 - val_accuracy: 0.8653
Epoch 60/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4819 - accuracy: 0.9190 - val_loss: 0.8491 - val_accuracy: 0.8103
Epoch 61/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4738 - accuracy: 0.9217 - val_loss: 0.7046 - val_accuracy: 0.8580
Epoch 62/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4642 - accuracy: 0.9233 - val_loss: 0.6359 - val_accuracy: 0.8817
Epoch 63/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4484 - accuracy: 0.9289 - val_loss: 0.7756 - val_accuracy: 0.8333
Epoch 64/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4611 - accuracy: 0.9220 - val_loss: 0.8272 - val_accuracy: 0.8133
Epoch 65/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4490 - accuracy: 0.9272 - val_loss: 0.8940 - val_accuracy: 0.7983
Epoch 66/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4591 - accuracy: 0.9229 - val_loss: 0.6173 - val_accuracy: 0.8877
Epoch 67/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4533 - accuracy: 0.9277 - val_loss: 0.7735 - val_accuracy: 0.8337
Epoch 68/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4451 - accuracy: 0.9277 - val_loss: 0.6264 - val_accuracy: 0.8797
Epoch 69/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4468 - accuracy: 0.9255 - val_loss: 0.7092 - val_accuracy: 0.8557
Epoch 70/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4344 - accuracy: 0.9312 - val_loss: 0.6534 - val_accuracy: 0.8717
Epoch 71/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4333 - accuracy: 0.9289 - val_loss: 0.6506 - val_accuracy: 0.8750
Epoch 72/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4319 - accuracy: 0.9308 - val_loss: 0.6790 - val_accuracy: 0.8630
Epoch 73/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4364 - accuracy: 0.9282 - val_loss: 0.6894 - val_accuracy: 0.8587
Epoch 74/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4151 - accuracy: 0.9373 - val_loss: 0.7134 - val_accuracy: 0.8530
Epoch 75/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4237 - accuracy: 0.9319 - val_loss: 0.7373 - val_accuracy: 0.8410
Epoch 76/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4229 - accuracy: 0.9351 - val_loss: 0.6581 - val_accuracy: 0.8753
Epoch 77/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4185 - accuracy: 0.9346 - val_loss: 0.6240 - val_accuracy: 0.8773
Epoch 78/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4143 - accuracy: 0.9363 - val_loss: 0.8559 - val_accuracy: 0.8207
Epoch 79/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4102 - accuracy: 0.9363 - val_loss: 0.7102 - val_accuracy: 0.8537
Epoch 80/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4060 - accuracy: 0.9380 - val_loss: 0.8016 - val_accuracy: 0.8247
Epoch 81/200
448/448 [==============================] - 2s 4ms/step - loss: 0.4066 - accuracy: 0.9377 - val_loss: 1.0908 - val_accuracy: 0.7640
Epoch 82/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3990 - accuracy: 0.9375 - val_loss: 0.6526 - val_accuracy: 0.8757
Epoch 83/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3975 - accuracy: 0.9409 - val_loss: 0.5463 - val_accuracy: 0.9033
Epoch 84/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3933 - accuracy: 0.9427 - val_loss: 0.8883 - val_accuracy: 0.8057
Epoch 85/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3993 - accuracy: 0.9368 - val_loss: 0.5707 - val_accuracy: 0.8950
Epoch 86/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3775 - accuracy: 0.9453 - val_loss: 0.5956 - val_accuracy: 0.8920
Epoch 87/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3831 - accuracy: 0.9447 - val_loss: 0.7074 - val_accuracy: 0.8593
Epoch 88/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3913 - accuracy: 0.9419 - val_loss: 0.7366 - val_accuracy: 0.8473
Epoch 89/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3815 - accuracy: 0.9444 - val_loss: 0.7926 - val_accuracy: 0.8400
Epoch 90/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3803 - accuracy: 0.9442 - val_loss: 0.6493 - val_accuracy: 0.8710
Epoch 91/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3774 - accuracy: 0.9447 - val_loss: 0.6531 - val_accuracy: 0.8730
Epoch 92/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3726 - accuracy: 0.9475 - val_loss: 0.6629 - val_accuracy: 0.8747
Epoch 93/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3637 - accuracy: 0.9495 - val_loss: 0.6954 - val_accuracy: 0.8670
Epoch 94/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3714 - accuracy: 0.9453 - val_loss: 0.6606 - val_accuracy: 0.8733
Epoch 95/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3794 - accuracy: 0.9446 - val_loss: 0.5996 - val_accuracy: 0.8823
Epoch 96/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3695 - accuracy: 0.9487 - val_loss: 0.6365 - val_accuracy: 0.8813
Epoch 97/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3664 - accuracy: 0.9488 - val_loss: 0.7054 - val_accuracy: 0.8597
Epoch 98/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3690 - accuracy: 0.9462 - val_loss: 0.8169 - val_accuracy: 0.8353
Epoch 99/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3700 - accuracy: 0.9467 - val_loss: 0.6534 - val_accuracy: 0.8733
Epoch 100/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3654 - accuracy: 0.9499 - val_loss: 0.7115 - val_accuracy: 0.8597
Epoch 101/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3586 - accuracy: 0.9502 - val_loss: 0.6737 - val_accuracy: 0.8707
Epoch 102/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3527 - accuracy: 0.9526 - val_loss: 0.6915 - val_accuracy: 0.8663
Epoch 103/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3565 - accuracy: 0.9489 - val_loss: 0.5873 - val_accuracy: 0.8943
Epoch 104/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3585 - accuracy: 0.9511 - val_loss: 0.7842 - val_accuracy: 0.8480
Epoch 105/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3476 - accuracy: 0.9520 - val_loss: 0.5950 - val_accuracy: 0.9010
Epoch 106/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3435 - accuracy: 0.9529 - val_loss: 0.6358 - val_accuracy: 0.8833
Epoch 107/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3467 - accuracy: 0.9516 - val_loss: 0.6223 - val_accuracy: 0.8857
Epoch 108/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3410 - accuracy: 0.9534 - val_loss: 0.5739 - val_accuracy: 0.8983
Epoch 109/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3455 - accuracy: 0.9530 - val_loss: 0.6075 - val_accuracy: 0.8960
Epoch 110/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3543 - accuracy: 0.9510 - val_loss: 0.6507 - val_accuracy: 0.8803
Epoch 111/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3470 - accuracy: 0.9530 - val_loss: 0.7828 - val_accuracy: 0.8347
Epoch 112/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3398 - accuracy: 0.9537 - val_loss: 0.6386 - val_accuracy: 0.8780
Epoch 113/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3311 - accuracy: 0.9573 - val_loss: 0.6460 - val_accuracy: 0.8793
Epoch 114/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3304 - accuracy: 0.9574 - val_loss: 0.7489 - val_accuracy: 0.8483
Epoch 115/200
448/448 [==============================] - 2s 5ms/step - loss: 0.3465 - accuracy: 0.9517 - val_loss: 0.8471 - val_accuracy: 0.8257
Epoch 116/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3340 - accuracy: 0.9569 - val_loss: 0.5736 - val_accuracy: 0.8993
Epoch 117/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3209 - accuracy: 0.9617 - val_loss: 0.6285 - val_accuracy: 0.8787
Epoch 118/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3277 - accuracy: 0.9550 - val_loss: 0.6938 - val_accuracy: 0.8620
Epoch 119/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3251 - accuracy: 0.9571 - val_loss: 0.6899 - val_accuracy: 0.8687
Epoch 120/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3277 - accuracy: 0.9570 - val_loss: 0.7505 - val_accuracy: 0.8543
Epoch 121/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3258 - accuracy: 0.9562 - val_loss: 0.7509 - val_accuracy: 0.8583
Epoch 122/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3198 - accuracy: 0.9605 - val_loss: 0.6488 - val_accuracy: 0.8800
Epoch 123/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3186 - accuracy: 0.9586 - val_loss: 0.7425 - val_accuracy: 0.8467
Epoch 124/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3232 - accuracy: 0.9584 - val_loss: 0.7319 - val_accuracy: 0.8563
Epoch 125/200
448/448 [==============================] - 2s 5ms/step - loss: 0.3237 - accuracy: 0.9582 - val_loss: 0.5739 - val_accuracy: 0.9003
Epoch 126/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3241 - accuracy: 0.9588 - val_loss: 0.6832 - val_accuracy: 0.8670
Epoch 127/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3214 - accuracy: 0.9577 - val_loss: 0.6590 - val_accuracy: 0.8823
Epoch 128/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3181 - accuracy: 0.9600 - val_loss: 0.7693 - val_accuracy: 0.8490
Epoch 129/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3081 - accuracy: 0.9606 - val_loss: 0.6948 - val_accuracy: 0.8633
Epoch 130/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3166 - accuracy: 0.9588 - val_loss: 0.5903 - val_accuracy: 0.8947
Epoch 131/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3092 - accuracy: 0.9611 - val_loss: 0.6937 - val_accuracy: 0.8653
Epoch 132/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3129 - accuracy: 0.9610 - val_loss: 0.6722 - val_accuracy: 0.8710
Epoch 133/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3033 - accuracy: 0.9633 - val_loss: 0.6687 - val_accuracy: 0.8780
Epoch 134/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3074 - accuracy: 0.9601 - val_loss: 0.6904 - val_accuracy: 0.8703
Epoch 135/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3177 - accuracy: 0.9571 - val_loss: 0.7082 - val_accuracy: 0.8660
Epoch 136/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2997 - accuracy: 0.9638 - val_loss: 0.9495 - val_accuracy: 0.8080
Epoch 137/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3150 - accuracy: 0.9599 - val_loss: 0.7511 - val_accuracy: 0.8533
Epoch 138/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3015 - accuracy: 0.9625 - val_loss: 0.6773 - val_accuracy: 0.8810
Epoch 139/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2942 - accuracy: 0.9662 - val_loss: 0.6151 - val_accuracy: 0.8930
Epoch 140/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2982 - accuracy: 0.9650 - val_loss: 0.8528 - val_accuracy: 0.8407
Epoch 141/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3032 - accuracy: 0.9609 - val_loss: 0.8641 - val_accuracy: 0.8240
Epoch 142/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3108 - accuracy: 0.9596 - val_loss: 0.7643 - val_accuracy: 0.8483
Epoch 143/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2958 - accuracy: 0.9651 - val_loss: 0.8073 - val_accuracy: 0.8467
Epoch 144/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2953 - accuracy: 0.9670 - val_loss: 0.7977 - val_accuracy: 0.8440
Epoch 145/200
448/448 [==============================] - 2s 4ms/step - loss: 0.3041 - accuracy: 0.9609 - val_loss: 0.5829 - val_accuracy: 0.8963
Epoch 146/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2995 - accuracy: 0.9628 - val_loss: 0.6626 - val_accuracy: 0.8833
Epoch 147/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2975 - accuracy: 0.9645 - val_loss: 0.6295 - val_accuracy: 0.8850
Epoch 148/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2884 - accuracy: 0.9667 - val_loss: 0.8552 - val_accuracy: 0.8383
Epoch 149/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2973 - accuracy: 0.9639 - val_loss: 1.0346 - val_accuracy: 0.7873
Epoch 150/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2938 - accuracy: 0.9637 - val_loss: 0.6316 - val_accuracy: 0.8830
Epoch 151/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2883 - accuracy: 0.9660 - val_loss: 0.8132 - val_accuracy: 0.8480
Epoch 152/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2835 - accuracy: 0.9677 - val_loss: 0.6450 - val_accuracy: 0.8783
Epoch 153/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2903 - accuracy: 0.9631 - val_loss: 0.4856 - val_accuracy: 0.9220
Epoch 154/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2829 - accuracy: 0.9668 - val_loss: 0.5849 - val_accuracy: 0.8987
Epoch 155/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2872 - accuracy: 0.9654 - val_loss: 0.7318 - val_accuracy: 0.8617
Epoch 156/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2942 - accuracy: 0.9620 - val_loss: 0.6844 - val_accuracy: 0.8700
Epoch 157/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2813 - accuracy: 0.9671 - val_loss: 0.8395 - val_accuracy: 0.8443
Epoch 158/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2885 - accuracy: 0.9655 - val_loss: 0.8384 - val_accuracy: 0.8367
Epoch 159/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2923 - accuracy: 0.9636 - val_loss: 0.7600 - val_accuracy: 0.8593
Epoch 160/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2837 - accuracy: 0.9675 - val_loss: 0.6015 - val_accuracy: 0.8980
Epoch 161/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2737 - accuracy: 0.9696 - val_loss: 0.6319 - val_accuracy: 0.8870
Epoch 162/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2835 - accuracy: 0.9661 - val_loss: 0.8353 - val_accuracy: 0.8430
Epoch 163/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2777 - accuracy: 0.9666 - val_loss: 0.7836 - val_accuracy: 0.8573
Epoch 164/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2822 - accuracy: 0.9661 - val_loss: 0.7182 - val_accuracy: 0.8680
Epoch 165/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2753 - accuracy: 0.9682 - val_loss: 0.7819 - val_accuracy: 0.8507
Epoch 166/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2725 - accuracy: 0.9687 - val_loss: 0.7533 - val_accuracy: 0.8680
Epoch 167/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2798 - accuracy: 0.9651 - val_loss: 1.1684 - val_accuracy: 0.7690
Epoch 168/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2829 - accuracy: 0.9667 - val_loss: 0.7680 - val_accuracy: 0.8633
Epoch 169/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2811 - accuracy: 0.9671 - val_loss: 0.6153 - val_accuracy: 0.8910
Epoch 170/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2730 - accuracy: 0.9691 - val_loss: 0.7026 - val_accuracy: 0.8767
Epoch 171/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2699 - accuracy: 0.9691 - val_loss: 0.7009 - val_accuracy: 0.8747
Epoch 172/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2672 - accuracy: 0.9717 - val_loss: 0.6144 - val_accuracy: 0.8910
Epoch 173/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2770 - accuracy: 0.9666 - val_loss: 0.8084 - val_accuracy: 0.8493
Epoch 174/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2746 - accuracy: 0.9703 - val_loss: 0.6553 - val_accuracy: 0.8867
Epoch 175/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2738 - accuracy: 0.9689 - val_loss: 0.6026 - val_accuracy: 0.8947
Epoch 176/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2765 - accuracy: 0.9667 - val_loss: 0.6372 - val_accuracy: 0.8850
Epoch 177/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2694 - accuracy: 0.9709 - val_loss: 0.6582 - val_accuracy: 0.8830
Epoch 178/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2664 - accuracy: 0.9714 - val_loss: 0.6913 - val_accuracy: 0.8717
Epoch 179/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2698 - accuracy: 0.9689 - val_loss: 0.8218 - val_accuracy: 0.8460
Epoch 180/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2666 - accuracy: 0.9717 - val_loss: 0.8104 - val_accuracy: 0.8490
Epoch 181/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2644 - accuracy: 0.9700 - val_loss: 0.5540 - val_accuracy: 0.9030
Epoch 182/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2660 - accuracy: 0.9716 - val_loss: 0.5937 - val_accuracy: 0.8973
Epoch 183/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2672 - accuracy: 0.9717 - val_loss: 0.6381 - val_accuracy: 0.8863
Epoch 184/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2684 - accuracy: 0.9680 - val_loss: 0.8586 - val_accuracy: 0.8347
Epoch 185/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2779 - accuracy: 0.9670 - val_loss: 0.6262 - val_accuracy: 0.8927
Epoch 186/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2621 - accuracy: 0.9737 - val_loss: 0.5417 - val_accuracy: 0.9150
Epoch 187/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2683 - accuracy: 0.9705 - val_loss: 0.6200 - val_accuracy: 0.8890
Epoch 188/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2568 - accuracy: 0.9726 - val_loss: 0.7144 - val_accuracy: 0.8677
Epoch 189/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2627 - accuracy: 0.9702 - val_loss: 0.8036 - val_accuracy: 0.8500
Epoch 190/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2664 - accuracy: 0.9694 - val_loss: 0.8031 - val_accuracy: 0.8573
Epoch 191/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2636 - accuracy: 0.9714 - val_loss: 0.7754 - val_accuracy: 0.8553
Epoch 192/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2592 - accuracy: 0.9731 - val_loss: 0.7465 - val_accuracy: 0.8690
Epoch 193/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2573 - accuracy: 0.9724 - val_loss: 0.7393 - val_accuracy: 0.8580
Epoch 194/200
448/448 [==============================] - 2s 5ms/step - loss: 0.2632 - accuracy: 0.9692 - val_loss: 0.7538 - val_accuracy: 0.8610
Epoch 195/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2594 - accuracy: 0.9715 - val_loss: 0.6934 - val_accuracy: 0.8720
Epoch 196/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2563 - accuracy: 0.9719 - val_loss: 0.7227 - val_accuracy: 0.8697
Epoch 197/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2557 - accuracy: 0.9738 - val_loss: 0.5621 - val_accuracy: 0.9063
Epoch 198/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2553 - accuracy: 0.9726 - val_loss: 0.7062 - val_accuracy: 0.8640
Epoch 199/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2570 - accuracy: 0.9714 - val_loss: 0.5510 - val_accuracy: 0.9067
Epoch 200/200
448/448 [==============================] - 2s 4ms/step - loss: 0.2566 - accuracy: 0.9708 - val_loss: 0.5760 - val_accuracy: 0.9013

plot_learning_curve(Conv2D_31_aug_l2_V2_history.history)

Observations

• The learning curves look better than the previous model but it is still unable to converge as well as the unaugmented data, hence I will use the unaugmented data as the final model.

Conv2D_31_aug_l2_V2.evaluate(test_data_31.batch(32))

94/94 [==============================] - 1s 4ms/step - loss: 0.6151 - accuracy: 0.8893

[0.6150560975074768, 0.8893333077430725]

Observations

• Accuracy of 88% is quite high but still not as high as the unaugmented data.

Final Models

• Based on observations of the learning curve, I have picked my two final models to classify 31 x 31 images and 128 x 128 images.
  • 31 x 31 image: L2 Regularizer Conv2D
  • 128 x 128 image: L2 Regularizer Conv2D with Augmentation

# Save the weights for 31 x 31
Conv2D_31_L2.save_weights("./CNN_FinalModels/Final_31x31.h5")

# Save the weights for 128 x 128
Conv2D_128_improved_aug.save_weights("./CNN_FinalModels/Final_128x128.h5")

Model Evaluation

31 x 31 images

# Create a list of images and labels
data_split = [(img.numpy(), classes[labels]) for img, labels in test_data_31]

# Convert the list to numpy arrays
X_test_31 = np.array([img for img, label in data_split])
y_test_31 = np.array([label for img, label in data_split])

y_test_int_31 = np.array([class_to_index[label] for label in y_test_31])
print(X_test_31.shape)
print(y_test_int_31.shape)

(3000, 31, 31, 1)
(3000,)

y_pred_31 = Conv2D_31_L2.predict(X_test_31)

94/94 [==============================] - 0s 5ms/step

report_31 = classification_report(
    y_test_int_31, 
    np.argmax(y_pred_31, axis=1), 
    target_names=classes)
print(report_31)

              precision    recall  f1-score   support

        Bean       0.95      0.94      0.94       200
Bitter_Gourd       0.95      0.94      0.94       200
Bottle_Gourd       0.97      0.99      0.98       200
     Brinjal       0.92      0.94      0.93       200
    Broccoli       0.94      0.95      0.95       200
     Cabbage       0.93      0.92      0.92       200
    Capsicum       0.95      0.95      0.95       200
      Carrot       0.98      0.89      0.93       200
 Cauliflower       0.90      0.94      0.92       200
    Cucumber       0.92      0.98      0.95       200
      Papaya       0.94      0.94      0.94       200
      Potato       0.90      0.90      0.90       200
     Pumpkin       0.91      0.95      0.93       200
      Radish       0.97      0.92      0.94       200
      Tomato       0.89      0.88      0.88       200

    accuracy                           0.93      3000
   macro avg       0.94      0.93      0.93      3000
weighted avg       0.94      0.93      0.93      3000

Observations

• Based on the other score metrics like precision, recall and f1, the scores seem to show that the vegetables are being identified quite well.

# Display the confusion matrix
cm_31 = confusion_matrix(y_test_int_31, np.argmax(y_pred_31, axis=1))

cm_df = pd.DataFrame(cm_31, index=classes, columns=classes)
plt.figure(figsize=(10, 6))
sns.heatmap(cm_df, annot=True, cmap="Blues", fmt="d")
plt.title("Confusion Matrix 31 x 31")
plt.ylabel("True Label")
plt.xlabel("Predicted Label")
plt.show()

Observations:

• Based on observations, it seems that cabbage and cauliflower are getting mixed up, as well as carrot and potato. Most likely the lower pixels caused some details to be lost.
• Cabbage and cauliflower have similar sizes and the lower pixels caused details like shapes and textures to be lost.

def PrintSampleImages(model, test_data, num_samples=5):
    # Get predictions for a batch of test images
    img, label = iter(test_data).next()
    predictions = model.predict(img)

    # Plot the sample images along with their predictions
    plt.figure(figsize=(15, 3))
    for i in range(num_samples):
        plt.subplot(1, num_samples, i + 1)
        plt.imshow(img[i], cmap='gray')
        plt.title(f"Predicted: {classes[np.argmax(predictions[i])]}")
        plt.axis("off")
    plt.show()

sample_images = PrintSampleImages(model=Conv2D_31_L2, test_data=test_data_31.batch(10))

1/1 [==============================] - 0s 15ms/step

incorrect_indices_31 = np.where(np.argmax(y_pred_31, axis=1) != y_test_int_31)[0]

# Randomly select a few incorrect predictions for visualisation
num_samples = min(5, len(incorrect_indices_31))
selected_indices_31 = np.random.choice(incorrect_indices_31, size=num_samples, replace=False)

#Visualise the images
plt.figure(figsize=(15, 5))

for i, index in enumerate(selected_indices_31):
    plt.subplot(1, num_samples, i + 1)
    plt.imshow(X_test_31[index], cmap='gray')
    plt.title(f"True: {classes[y_test_int_31[index]]}\nPredicted: {classes[np.argmax(y_pred_31, axis=1)[index]]}")
    plt.axis("off")

Observations

• As such, the lower pixels caused the textures to be lost, causing images like radish and cucumber to be mixed up.

128 x 128 images

# Create a list of images and labels
data_split = [(img.numpy(), classes[labels]) for img, labels in test_data_128]

# Convert the list to numpy arrays
X_test_128 = np.array([img for img, label in data_split])
y_test_128 = np.array([label for img, label in data_split])

y_test_int_128 = np.array([class_to_index[label] for label in y_test_128])
print(X_test_128.shape)
print(y_test_128.shape)

(3000, 128, 128, 1)
(3000,)

y_pred_128 = Conv2D_128_improved_aug.predict(X_test_128)

94/94 [==============================] - 1s 3ms/step

report_128 = classification_report(
    y_test_int_128, 
    np.argmax(y_pred_128, axis=1), 
    target_names=classes)
print(report_128)

              precision    recall  f1-score   support

        Bean       0.97      0.97      0.97       200
Bitter_Gourd       1.00      0.92      0.96       200
Bottle_Gourd       0.93      0.98      0.96       200
     Brinjal       0.86      0.94      0.90       200
    Broccoli       0.93      0.97      0.95       200
     Cabbage       0.91      0.94      0.93       200
    Capsicum       0.99      0.96      0.98       200
      Carrot       0.98      0.94      0.96       200
 Cauliflower       0.94      0.94      0.94       200
    Cucumber       0.99      0.95      0.97       200
      Papaya       0.95      0.95      0.95       200
      Potato       0.97      0.94      0.96       200
     Pumpkin       0.94      0.93      0.93       200
      Radish       0.99      0.94      0.96       200
      Tomato       0.90      0.94      0.92       200

    accuracy                           0.95      3000
   macro avg       0.95      0.95      0.95      3000
weighted avg       0.95      0.95      0.95      3000

Observations

• Based on the other score metrics like precision, recall and f1, the scores seem to show that the vegetables are being identified quite well.

# Display the confusion matrix
cm_128 = confusion_matrix(y_test_int_128, np.argmax(y_pred_128, axis=1))

cm_df = pd.DataFrame(cm_128, index=classes, columns=classes)
plt.figure(figsize=(10, 6))
sns.heatmap(cm_df, annot=True, cmap="Blues", fmt="d")
plt.title("Confusion Matrix 128 x 128")
plt.ylabel("True Label")
plt.xlabel("Predicted Label")
plt.show()

Observations:

• Vegetables like Brinjal and Potato are easy for the CNN to mix up as they are shaped quite similarly, hence it has the most labels getting mixed up.
• Otherwise, the model is able to predict most labels accurately.

sample_images = PrintSampleImages(model=Conv2D_128_improved_aug, test_data=test_data_128.batch(10))

1/1 [==============================] - 0s 16ms/step

Error Analysis

• We will only inspect the 128 x 128 images as it is clearer for us to visualise what went wrong.

incorrect_indices_128 = np.where(np.argmax(y_pred_128, axis=1) != y_test_int_128)[0]

# Randomly select a few incorrect predictions for visualisation
num_samples = min(5, len(incorrect_indices_128))
selected_indices_128 = np.random.choice(incorrect_indices_128, size=num_samples, replace=False)

#Visualise the images
plt.figure(figsize=(15, 5))

for i, index in enumerate(selected_indices_128):
    plt.subplot(1, num_samples, i + 1)
    plt.imshow(X_test_128[index], cmap='gray')
    plt.title(f"True: {classes[y_test_int_128[index]]}\nPredicted: {classes[np.argmax(y_pred_128, axis=1)[index]]}")
    plt.axis("off")

Observations

• Based on the images, we can see that vegetables like radish is very hard to predict correctly without the use of colors, same goes with the others.
• Shapes of the other vegetables like papaya and potato are also quite similar hence the CNN model will also think the same way.
• I am not sure why papaya is even considered a vegetable in the dataset but it is.

5) Summary

In summary, when training CNN models, regularization techniques like L2 and Dropout are very efficient when dealing with images of smaller batch sizes. Data augmentation does help the model's accuracy to a certain extent but it depends on case-to-case scenarios. Room for improvement can be made to the images like trying image segmentation or even trying out other models like VGG to classify the vegetables.